Higher Hard Drive Prices, Still The New Normal

About 7 months ago, back at the end of Q1 2012 we researched hard drive prices and compared them to pre-flood levels. We concluded that higher prices would be here to stay for the next few years for a couple of reasons:

• Nobody is investing in next-generation facilities with only 2 serious players in the spinning rust market, there is no need reduce manufacturing cost in the medium term with next generation factories or processes.

• A lot of the specialist tools and precision equipment used for testing and building hard drives are no-longer available. The firms that made them have long since gone bust as the market consolidated in the last 15 years. Lack of such equipment is a barrier to increasing production and (re)building plants.

• While Solid State Disks continue to march into mobile, desktop, and enterprise storage, the bulk of new added capacity will continue to be provided by spinning disks. In the enterprise data centre they form either a thin tier or are deployed tactically to take care of IOPS/latency hot-spots. Hard disks in the enterprise are not about to be wiped out by SSD storage.

The bad news is that the majority of drives are still more expensive now than pre-flood. Some are around 60-70% more expensive than pre-flood levels.

If your organisation or project is hamstrung by storage costs or performance, talk to us. We have helped companies in Finance, Life Science, Academic, and Software sectors accomplish and exceed their goals, on-time and on-budget, in spite of rising storage costs. Get in touch to find out how.

Why VMware (still) Wont Convince The Cloud Providers

We don’t normally comment on commentary at the 360is blog. Most of our postings are more substantial, whitepapers, guides, seminars, or software we’ve produced or packaged. However there is a good posting (Cloud Hosting & Service Provider Forum) by Richard Talaber (ex-CTO’s office at VMware) on LinkedIn and thought I’d reflect on it here.

For those of you who aren’t on LinkedIn, or don’t want to join the group in question, Richard was responding to Beth Pariseau’s article at TechTarget on VMware’s recent abolition of the vRAM tax.

Richard makes the case that VMware is probably not going to be the right virtualisation stack for a commodity “$75” per-month per-VM hosting/cloud service, vRAM tax or not. He also believes that VMware still makes sense for the smaller market for high-value fully-managed VM hosting, with load balancing, monitoring, fault tolerance, and high performance throughput bells and whistles.

Mostly I agree with Richard, he makes sensible arguments supported by good assumptions but in-spite of this I still see more than a few problems for VMware in the short and medium term. Let me explain...

All service providers, including Cloud & Hosting providers, need to own their own infrastructure if they are to have a sustainable business. Whether it be a fibre optic network (in the case of a carriers) or their provisioning, monitoring, and management stack in the case of Cloud providers. Unless a service provider owns his own infrastructure, he cannot exercise full control over his costs. Cost control is vital in a service provider business, as profits depends upon volume, and any tax on profits is unwelcome, especially one that grows in-line with that volume.

Hardware must be paid for (hopefully using inexpensive long-term debt secured against assets you also own), but if there is even a chance of finding a cheap or inexpensive hypervisor-and-management-stack then Cloud providers have to take it. This is why you see so many of them with Xen, XenServer or KVM. This is why the list of users of OpenStack reads like a who’s who of Telecoms and Hosting. For those that argue I'm not considering TCO, think about this. I've hired good technical guys, pay them well, they work hard for me, I'm supposed to be a player in Cloud. If my guys can’t engineer something solid, maintainable, and cost effective then what am I doing in this business? These guys and the platform they build from KVM, Xen, or whatever, are my long-term competitive advantage. At least until I'm big enough to be building my own data centers from scratch.

If I'm a Cloud provider and my maintenance renewal (or any other per-unit-customer cost) jumps even 1%, its a big deal, I've got a bazillion systems after all.

For parts of the infrastructure that can’t be wholly owned, or cannot be had for free (with effort from my hardworking DevOps guys), the Cloud provider's only weapon is over-subscription. Pay for the product then figure out how to dilute that cost by maximising over-subscription, balance providing a great service with running the hypervisor and server hot. It is a very difficult balance. This is one of the reasons the vRAM tax was so wildly unpopular with Cloud providers, it took away one of their profit levers, or at the very least shortened it.

Hosting/Cloud providers are currently at a very immature stage in life, they are talking about RAM, and IOPS, and vCPUs or is it CPUs or is it Cores or Threads? What do any of those things mean? What about transfer rates, or latency, what kind of cores? What is a thread? These things don't mean much to business people so good luck trying to explain them. When business sees 3 prices for what they perceive as essentially the same service, they are going to go for the cheapest and find out later if it was appropriate. By Richard’s own admission, 80% of workloads are relatively modest in demands. VMware will not manage to get across an argument based upon performance; a performance lead of the size VMware achieves (or even aspires to achieve) is not a sustainable advantage for the Cloud provider market. 

Richard's last sentence is key: "Perhaps VMware should consider a public cloud price that is significantly less expensive than a private cloud price.". Logical. If the public cloud VMware price were somehow so small as to barely matter, then service providers would not spend time fiddling around with the competition or knocking up free alternative platforms....but how to segment the product? I think it is too late now, there are alternatives in-use, there are engineers out there with experience of building these infrastructures for service providers, and even if you can't afford to buy such people you can probably rent them. Talk to us.

• Better performance is not a sustainable advantage for VMware.
• A richer feature set probably isn’t either.
• Cloud providers dislike anything that handicaps their ability to over-subscribe.
• Cost that grows in-line with customer volume is a no-no, unless it is absolutely impossible to avoid.

VMware could make their product more appealing to commodity Cloud providers, but in order to do so they’ll have to start thinking more like them. Or talk to someone who does. 360is has helped companies like CheckPoint, HP, and Microsoft understand the Cloud service provider market. You know where to find us.

FreeBSD 9.0, Paravirtualized Drivers, & Xen-Tools On XenServer 6

We are often asked by clients to help them squeeze more performance from an existing infrastructure, to speed up an application or shorten an IT-dependent business process. Early on in the discussion there's a tendency for sysadmins and architects to dive in to technical minutiae, the black magic and chicken-waving of memory page sizes, block alignment, cache segmentation, and thread pinning. This is almost always a mistake at this early stage and can drain days and weeks with few results. The business gets frustrated when deadlines are missed and the techies can't answer simple questions like "when will this be fixed?", "how much sooner will I get my results?", and "what will it cost to process data within that time window?". Black magic has a time and a place, but at 360is we save our voodoo until the latter stages of a performance tuning project.

Before making non-portable, hard-to-maintain, unsupported, obscure, or fragile configuration changes that break when you least expect them (and may not even be understood by those operating the infrastructure) we start with the basics:

• Is what you have, setup properly? Many complex multi-vendor data centres have a setup that is sub-optimal in some way.
• Is the system already performing more-or-less as you would expect "on paper" +/-25%? We whiteboard a block diagram of devices, buses, networks, data volumes, benchmarks.
• If it isn't then there is probably a significant configurational mistake to be found. Forget about jumbo frames if your switch is stuck in half-duplex mode.
• Focus on the biggest wins first, they are often the easiest to achieve. Only then do we go chasing marginal gains with our consultant's Juju. Don't waste days chasing the last 3% unless that gain makes economic sense for your business process.

Where are we going with all this and what does it have to do with FreeBSD?

At 360is we are fans of FreeBSD, and regularly recommend it as a secure, low-maintenance, stable, and performant, server operating system. Unfortunately there is no official support for FreeBSD in the Citrix XenServer product. What this means in practice is that those wanting to run FreeBSD on XenServer deploy it in HVM mode and without Xen-Tools. Pure HVM mode is slow for network and disk access, and no Xen-Tools means no live migration, not good for production workloads.

Shooting for the biggest win first, we have made available a basic, paravirtualized-drivers-with-xentools-installed, FreeBSD 9.0 64-bit Template that delivers approximately twice the performance of the pure HVM install of the Operating System. No tuning, no special settings, absolutely no chicken-waving.

• FreeBSD 9.0
• 64-Bit (amd64)
• Paravirtualized Drivers in the XENHVM kernel
• Open Source Xen-Tools pre-installed
• Small (389MB) XVA file

Bugs/Errata: Hot-adding of additional NICs works, hot-remove not so much. Use at your own risk. Please submit any other problems as comments, feedback is always welcome.

Get the FreeBSD 9.0 XenServer Template XVA, (cookies/valid Email required in order to be sent the password).

If you need to extract more performance or reliability from an existing infrastructure, application, or IT-driven process, 360is consultants know how, get in touch.

=== Update 26-07-12 ===
As we always get asked about these "relative-to" bar charts, the absolute figures were 188MB/sec for the VM derived from our template, and 92MB/sec for the ordinary install using Citrix "Other OS" template. The fast VM consumed 67% of 1 vCPU on an otherwise idle system. The physical hardware used was a VMCo VA12xx Appliance with its local IOPS sink configured as an SR.

=== Update 11-10-12 ===
This XVA was prepared for XenServer 6.0.2, and probably wont work on XenServer 6.1. If you have a commercial imperative for it on a different version of XenServer, with a different version of FreeBSD other than 9.0, or for that matter with i386 versus amd64, then get in touch with our project office.

The New Normal For Hard Drive Prices

Hard Drive Market Share (by units) & Consolidation CY2011

In our 2011 end of year message we touched on problems in the hard drive supply chain due to the serious flooding in Thailand starting late July 2011 and running through the rest of that year. At the time, the expectation would be that we would see prices return to normal in Q2 or 2012. It now looks like hard drive prices will not return to pre-flood levels. The evidence is that higher hard drive prices are "the new normal".

Consolidation in the hard drive industry, coupled with a prolonged period of low margins most likely means that prices will not return to their previous levels. So if you have been holding off on a storage upgrade, there is little incentive to keep waiting. The hard drive market has seen at least 2 rounds of mass extinction or consolidation over the last 20 years with tens of companies exiting the market, leaving just 3 manufacturers standing. Together Western Digital and Seagate have 87% of the market leaving Toshiba a distant 3rd. Nobody is investing $500M to build a new factory in order to lower their manufacturing costs and put one over on the other 2 guys.

While the take-up of SSDs continues in mobile devices (laptop, tablet), and they make an appearance in the enterprise as specialist devices or as a thin tier above enterprise SATA/SAS drives, "spinning rust" will continue form the bulk of all storage shipped in terms of Terabytes for as far out as anyone is brave enough to predict. Seagate has publicly stated it will be able to produce a 30-60TB 3.5-in. hard drive by 2020.

As an aside, anyone who has waited for a RAID5/6 rebuild on an array of full 1TB drives will know, increasing data density without increasing interface speed brings its own challenges, as does silent data corruption and the need to counter it . If you are worried about either of these problems then we can help you avoid them!

If you need to get more performance from your existing storage, or are struggling with data volumes, or if you simply require impartial advice ahead of making a purchase from one of the big storage vendors, we can probably help you. 360is has experience with environments of all sizes and have completed successful projects with most of the major vendors including  NetApp, EMC, Hitachi, HP, Sun/Oracle, Dell, 3PAR, Datacore, Westek, and Nexenta. Get in touch to get ahead of your storage problems.

360is Guide to Understanding, Commissioning, & Maximising Value from Penetration Testing or Security Assessments

-->

Clients often contact us while weighing up the value of getting a Security Assessment or Penetration Test. Whether it's a recent breach, compliance obligation, the regulator, or auditors that trigger the inquiry, we find ourselves repeating similar advice during those initial conversations. Their questions may be familiar to you;

• What exactly is a Penetration Test?
• Is it any different from a “scan” or a “vulnerability assessment”?
• What will it really do for us?
• What do we do with the results?
• How do I evaluate different companies offering this service?
• Why can’t I get a consistent budgetary cost from the market?

We've recorded the answers to these questions and more in one place, using consistent language,  in a way that can be understood by both IT and non-IT professionals alike. Whether you are a systems administrator, or a CSO (more likely in the UK, IT Director/Manager) you will be able to use this guide to reduce the time taken to protect your assets, meet your business needs, and keep the customer/auditor/regulator/boss happy.

Aren’t there already countless guides, papers, and articles on Penetration Testing and security? Certainly. However, most of them are years old, focused on (or written from) a non-UK perspective, or are difficult for non-technical readers to understand. Our guide is different.

-->

• UK & European perspective: While you can find an abundance of articles discussing Penetration Testing within the context of HIPAA, SOX, and FISMA, scarcely a nod is given to UK and European regulations and standards. Hackers may not respect geography, but your organisation still has to.

• Up to date: While technical details of vulnerabilities have changed, sysadmins, programmers, and engineers are still making many of the same mistakes now as when we did our first assessment in the mid 90s. However, language changes, an organisation’s view of IT changes, as do end-user working practices. This document reflects that, taking a contemporary view of the subject.

• Non-technical: Couched in ordinary terms the business can understand, this guide avoids much of the technical jargon that makes other articles heavy-reading for those whom IT security is not their full time occupation. While the skills employed may be highly technical, we can’t lose sight of the business problems being solved.

-->

We hope this guide will help many of our clients and future clients get the most from a Security Assessment/Penetration test (whether provided by 360is, our contemporaries, or your own IT security team):

Penetration Testing Guide, Part 1.

An Introduction to Penetration Testing. [PDF]

Penetration Testing Guide, Part 2.

Selecting A Penetration Testing Company. [PDF]

Penetration Testing Guide, Part 3.

Maximising Value From A Penetration Test. [PDF]

The Consolidated Penetration Testing Guide.

Parts 1,2, and 3 all in one document. [PDF]
Parts 1,2, and 3, text only, academic, no commentary. [PDF]

There will always be something missing from such a document; specific relevance to your particular situation. Get in touch to complete the picture. 360is is a company where you can talk to a client account manager who can get a consultant on the phone, without prior arrangement and without running the meter. Contact Us.

-----------------------------
 Update 24-05-2012
-----------------------------

For further information on 360is Penetration Testing Services, bookmark our Penetration Testing Homepage.

London 2012 Olympic Preparedness Checklist

At time of posting, there are exactly 100 days until the start of the London 2012 Summer Olympics. Are you ready for the starter's gun?

Whether you are a follower of the games or not, the numbers are impressive:

• 11,000 athletes will compete in 300 events accompanied by 7000 foreign officials
• 24,000 military and police will provide security, costing £500M
• 200,000 are employed by the London Olympic Committee
• 250,000 people will be at the Olympic park at any one time
• 500,000 international spectators will come to London
• 6,000,000 will visit London for the games in total
• 7,000,000 tickets will be sold

The Information Technology figures are similarly eye-opening:

• 60Gb/s of traffic will leave the venue during events
• The BBC will stream 1Tb/s of content to end users at peak times, 5-10x normal
• There are 1800 WIFI and 25 3G base-stations in the park
• 900 Acer servers, 11500 desktops and 1100 laptops
• 3,500 IT specialists will operate these systems including 400 desktop support technicians
• They will be fixing desktops and laptops running Microsoft Windows XP and Vista

The total cost for the games will be £12 billion, £2000 per visitor.

Along with the excitement and anticipation, thoughts have now turned firmly to practicalities of continuing business operations. While most of the official advice is around transport, security, and staffing, little has been said concerning IT and Telecommunications. Meanwhile our banking clients at Canary Wharf have already secured nearby hotel rooms for hundreds of key staff, one manufacturing company is providing static caravan's in their car park, another has taken the precaution of obtaining short-term office space outside the capital, complete with local IT. Our international clients began preparing immediately after London was announced, having learned from prior experience in Sydney.

If you are charged with keeping your organisation's technology running during the games, you now have just 70 working days to prepare, test, and put a plan into action. While a London Olympics is only a once-in-a-lifetime occurrence, planning for continued operations during the games shares much in common with other more frequent DR or BC challenges like the 2009 flu pandemic, the 2005 terrorist attacks, the 2005 fire, or the 2011 city-wide rioting, all of which are familiar to London businesses.

Even minor disruptions can mean you are denied access to your premises, your datacenter is without power, or the transport network fails. These "little disasters" happen every single day to someone, somewhere in the capital city, as we have already seen. If you haven't done anything yet it is not too late, a lot can be accomplished in 100 days but there is no time to lose.

360is have compiled a check list of things IT managers at London offices should think about before the Olympics. If you want to know how to cope with low bandwidth conditions in an emergency, provide IT service for remote users, tuning infrastructure to cope with VPN workers, or establishing a second datacentre outside London, get in touch.

Send me the 360is Olympic Preparedness Checklist

360is London Counter Terror Expo

Three Sixty will be present at the 2012 Counter Terror Expo, held at at The Grand Hall, Olympia, West Kensington. Between the 25th and 26th April we shall join with our clients and partners in the exhibition hall, briefings, and break-out sessions.

The event is open to the security industry, armed forces, government, equipment procurement organisations, specifiers, operators / end-users, trade media, and research establishments. There will be 9000 delegates and hundreds of exhibitors. Of the many streams and seminars to attend, we recommend that our clients and partners check the following:


Cyber Security & Electronic Terrorism
26th April 2012 (day 2)

08:00 - 08:45 Coffee/Registration
08:45 - 09:00 Chairman's Opening Remarks
09:00 - 09:30 Securing Cyberspace - Challenges and Consequences
09:30 - 10:00 Security and the Cyber threat
10:00 - 10:30 Cyber Attack Analysis
10:30 - 11:00 Coffee / Tea / Exhibition time

11:00 - 11:30 Policing Criminality in Cyberspace
11:30 - 11:55 Social Media - Friend, Foe, or Terrorist threat?
11:55 - 12:20 Corporate Espionage and Cyber Security
12:20 - 12:45 Cyber Security and the Threat to Information
12:45 - 14:00 Lunch / Exhibition time

14:00 - 14:25 Addressing Evolving Cyber Challenges
14:25 - 14:50 Multidimensional threats in the Mainstream
14:50 - 15:15 Smartphone Security Risks and Exploits
15:15 - 15:40 Countering Evolving and Emerging Cyber Security Threats
15:40 - 16:05 Cyber Security Preparedness in the UK
16:05 - 16:30 Panel Discussion
16:30 - 16:45 Close of Cyber Security Conference

Counter Terror Expo is a secure event and all visitor applications must pass through the security vetting procedure to assess their eligibility to attend. If you are unable to attend and would like to speak to one of our consultants about your information security project then please get in touch.

Update 25-04-12: We'll be wearing a red carnation and carrying a copy of the times.

April Seminar, Virtualisation In High-Tech Environments

101 Cambridge Science Park, Milton Rd

We are holding a free, invitation only seminar on the 23rd April 2012 for high-tech or research and development centric organisations in the Cambridge area.

Aimed at technical and business users with an interest in virtualisation, this is not a vendor-centric product briefing, but a sharing of real customer experiences in two demanding application areas and a detailed look at a world class software research and development facility. There will also be an opportunity to network with fellow members of the Cambridge technology cluster over lunch.

360is are hosting the event at Citrix’s R&D centre on the Cambridge Science Park. Two local case studies will feature, each from a different high-tech business that has adopted desktop or server virtualisation to increase productivity for specialist office workers, scientists, or engineers. If you are charged with supporting demanding, sophisticated users in the Cambridge area then this seminar will be of interest to you.

Contact the organiser for an invite if you would like to attend.

When: Monday April 23^rd 09:30 until 12:30

Where: Citrix Systems Inc, Building 101, Cambridge Science Park, Milton Road, Cambridge, CB4 0FY

Organiser: rob.gilson@360is.com

Summary Agenda:

09:30 Registration and Coffee

09:50 Start

10:00 Mark Heath VP of Products, Citrix XenServer

Detailed look at the engineering facilities and resources behind the Citrix R&D organisation

10:30 Case Study 1

Shortening the testing and development cycle for an large international software company

11:00 Break

11:15 Case Study 2

Virtual Desktop Infrastructure (VDI) for demanding, scientific, and engineering users

11:45 Closing remarks and prize draw

12:00 Buffet lunch and open networking 

About 360is

360is have over 10 years’ experience in providing consultancy and professional services in the application of  technology to business problems and challenges. Our staff have worked in and around Cambridge since the early 90s, solving problems for numerous Software, Life Sciences,  Electronics and R&D driven companies.

The Microsoft Virtual Desktop Licensing Conundrum

Typically, large established software companies take a long time to alter their licensing and pricing to accommodate new models of use or methods of deployment. Just as many companies initially refused to support or license their server software in a virtual environment, before eventually softening their position and in many cases re-working their license agreements to support or encourage server virtualisation, the same is true of VDI. The matter of reviewing licensing is made more difficult and slower by virtue of the fact that there is only 1 desktop Operating System provider who completely dominates the market.

While the importance of desktop OS choice is probably declining (in the face of application virtualisation, google apps, and tablet or embedded devices which to the user appear to “have no OS"), the case is currently that Microsoft completely controls the desktop. They feel less pressure to do anything with licensing than they did in the server market, where they had real competition.

360is recently reviewed the situation with Microsoft OS licensing for a Virtual Desktop Infrastructure service provider. Through our discussions with Microsoft, Citrix, and other software vendors we were able to unscramble the (current) conundrum around Microsoft desktop Operating System licensing in a VDI environment.

For an extract from the licensing section from our report "Multi-tenant Virtual Desktop Infrastructure Design" Read More.