Bladewatch.com

I was speaking with a delivery manager in the ‘blade team’ over at one of the multinationals and she was saying how the business had ‘taken over’ managing their blade farm from IT and ‘had transformed’ delivery, they had slashed costs and managed to vastly improve the end user perception of their blade farm.

I was intrigued and asked more about their achievements and why they felt IT was so poor at delivery. The statements ranged from inflexibility (taking too long and being ‘unresponsive’), to costs (internal cross charging), and being impacted by IT standards on delivery and process.   It was interesting getting the answers to the following questions:

Who prepared the operating system build?

IT adapted their build for our blades.

Who provided hardware support for the blades

An external service provider through our support contract.

Who prepared the data center environmental configuration?

IT and facilities managed the data center configuration and kitted it out to meet our requirements.

Who co-ordinated out of hours support and managed batch/workloads?

IT have a central team providing 24/7 monitoring of the infrastructure, our blades and the batch using monitoring tools.

Is the issue then, not that the blade team had transformed the way it did things, it had:

• Streamlined the blade build and created one unified voice using using IT as the layer to provide the core infrastructure
• That the blade team were still dependent on IT backing for the application and infrastructure monitoring and core infrastructure components

What we have to be careful of is that dangerous statement of ‘we can do it better’, yes with the difficult and ‘boring’ bits taken away and provided as a core service, we can absolutely do it better, but we have to remember that:

• IT has to be all things to all people
• IT standards are there for a reason
• IT budgets are always too high and delivery too poor

Without being emotional. The issue was not a success as much as it was failure. IT had failed in terms of relationships and branding. Both in terms of ‘establishing the blade team’, of catering to a specific business line or application need, and in selling its success at providing the core infrastructure which underpins the blade farm – let us not forget, without the data center your blades are of little benefit.

I wish we could see two things, IT being run more as a business with business backing rather than the often ‘cost’ debate combined with IT having a sense of branding and purpose. Would the business ever entertain one guy in accounts demanding that ‘his printer’ be a LaserJet 5SI because he’s had that printer and his deparment ‘isn’t paying for a new printer’, or put up with a developer demanding that his 8 year old server be upgraded to 1GB RAM rather than have a virtual machine or a new server because ‘it would take too long’ to migrate and test the code. Only because we accept this behaviour, because we set the benchmark do we let it continue and in doing so, damage our delivery and our revenues – a service provider would reply in both cases, fine fill your boots, but this is the true cost – oh you mean you didn’t mean it, fine we’ll deploy the new solution.

IT is a product, a service, a brand, the moment you as the IT and the business understand that and practice it, both in terms of reporting, delivering and managing, is the moment you transform your delivery, your business perception and by the nature of the game your process. Remember that when you deliver, no one really asks about costs, as soon though as people see an opportunity for head count, for budgets, it is then that we see:
Business alignment with IT – we can do it better but do you know the complexity of the IT infrastructure – and is it IT services you want a business line doing or is it earning revenue?
Cloud and outsourcing – but have you considered that the things you are able to do with your own IT would not be supported from an outsourced venture – sorry we don’t do that here, but sign this large disclaimer, and pay a fee and we might be able to. With internal IT, it costs you an email, “do you know who I am, I accept the risk, next.”

It takes to difficult things:

• Saying no to people
• Demanding more – better application code, better investment where necessary and enforcing the stability of production.
• Just doing it – no choice, no debate, there is a server, you have three weeks to move to it, because then we decommission your DL360 G2.

The most successful projects have been the ones where IT has been given the magic email, to which it can forward to when people say no:

“I don’t want a new pc, or a new monitor, I like my 22″ CRT display, I like my desktop it’s got all my files on it.”. Fine sir, absolutely sir, here’s an email from the CEO, you can raise it with him, in the meantime, the new pc and screen will be here on Friday between 3 and 4pm.

Share and Enjoy

I was speaking with one of my CIO friends today. He had asked for a survey of the server build process and of their storage room, where all the bits arrived and were kept. It was only because he had asked one of the managers to look at the way they build servers, reduce the delays and take a big picture view that they discovered:

• The servers were ordered and supplied with a CD ROM drive – but they then purchased a separate DVD drive as the operating system build was on dvd
• The servers often got ordered with an extra network card (if they were being used in a cluster or had specific requirements) – despite the server having 2 or four on board ports
• The servers were ordered with US power cables but they often either used the ones they had or purchased UK ones when required

As a result they had 127 cd rom drives, 93 sets of power cables and a number of network cards which had been purchased with the server. The CIO’s point? Where were we? Why had no one noticed and how much had we ‘wasted’ by not paying attention to detail?

The disconnect?

The purchasing team phoned the vendor and said “give me one of those servers”, the IT Manager/Project Manager quoted that it must “have 4GB RAM or 4 cpus, 146GB drives”, but they did not read through the quotation, they checked for processors, memory and drives. The ‘standard’ quotation, the ‘standard’ build and configuration included things which were not needed. Thus the company had spent time and effort, ordering components that were not needed and got either put in the bin, or stored ‘just in case’, just before he hung up he asked me:

“Want a propitiatory IDE cd drive, we’ve got 127 of them, we can throw in some power cables as well. Do you know they only admitted to me that the build is dvd based, very useful.?”

The server engineer assumed that someone knew the company was collecting hot swap cd rom drives, power cables and network cards. The manager followed the architecture teams specifications for servers and purchasing ordered what the vendor and purchasing had specified, “we need a server with a dvd drive”, UK power cables and another network card, it needs to be 4 processors, 16GB RAM and 8x 146GB hard drives”. The CIO just thought that the process worked, that purchasing would automatically look at the quote check what we purchased was needed, but they were ordering so many items, the server was just another one of those items, the thought the project manager, the business sponsor and the architect all knew what was being purchased.

We need to look at the big picture, combining the technology with the process, looking at what is delaying everything from the specification of requirements, the ordering and delivery, down to the operating system load and configuration.

Share and Enjoy

Burton Group

I would like to welcome the Burton Group to our blog (bladewatch.com), it is great that they have decided to support us in delivering our content to you. The Burton Group provide in-depth IT research and advisory services with a focus on strategic business technologies to executives and technologists. They have launched a new series of training videos which are informative, interesting to watch and crucially (as an end user) affordable, so do check them out, they cover topics including virtualization, data center economics and SharePoint.

Share and Enjoy

Earthtimes

SANTA FE, NM, November 17, 2009—UpsiteTechnologies, Inc., leading developer of highly engineered airflow solutions designed to optimize existing data center infrastructure is pleased to announce a partnership with Sensatronics, a leading source for temperature and environmental monitoring products for the IT industry.

For a limited time, Sensatronics will package Upsite’s award-winning HotLok Blanking Panels with its leading product, the Senturion. The HotLok Blanking Panel is engineered to seal openings in the front of the IT equipment racks to prevent hot air from circulating and the Senturion monitors temperature, environmental, and security conditions in the data center.

Both the HotLok product and the Senturion have been studied extensively and permit raising computer room temperature setpoints, which can save data centers thousands of dollars a year in operating costs (cooling energy reduction) and capital costs (deferred additional cooling infrastructure costs).

Increasing the temperature of the data center can reduce the energy costs substantially one of the CIOs I was speaking about was considering it and asking his colleagues how he could go about it, he gave me the example that raising it could save hundreds of thousands of pounds a year in energy cost, but only marginally increase his hardware support contract costs, either way it was still cheaper than running at the industry standard temperature. I’m off to check out more.

Share and Enjoy

Univa UD

Portland, OR, November 17, 2009 – Univa UD, a leading provider of cloud management software products, today announced UniCluster 5.0, a major new version of Univa’s award winning infrastructure and workload management software stack that provides the underpinnings of Univa’s cloud computing solutions.

UniCluster 5.0 features:

• Truly heterogeneous support: With UniCluster 5.0, a single cluster can support OpenSolaris and Linux compute nodes with a common management framework. Further, software profiles can be created and deployed for multiple operating systems (OS) and computing environments. Thus, users can create one Kit for deployment across resources running a variety of OS’s and also across multiple hardware profiles.
• Scalability: UniCluster 5.0 easily scales to over 7,000 nodes through subordinate installer nodes.
• Robust Web-based API: UniCluster offers a unique, well defined web-based API to allow interfacing with the UniCluster core framework.
• Integrated management console: System configuration is consolidated into a single management application.
• OpenSolaris support: UniCluster 5.0 is a native OpenSolaris version, functionally equivalent to the Linux

Being able to manage workloads more efficiently and use the same technology in order to manage cloud based solutions brings further service delivery and application performance management opportunities for the end users and IT alike. Anything we can do to remove the barriers of adopting cloud based technologies has to be a good thing for the end user community and cloud computing as a platform, I see it works with Sun Grid, very, very cool.

Share and Enjoy

HP.com

PALO ALTO, Calif., Nov. 17, 2009 – HP today announced the University of Minnesota Supercomputing Institute for Advanced Computational Research (MSI) has chosen HP blade servers for a new high-performance computing system to power research across a broad range of disciplines, including life and physical sciences and engineering.

The new HP system at MSI placed No. 67 on the recently published TOP500 list of the world’s top supercomputers, which HP ProLiant blade severs continue to lead with 42 percent of entries.

Powered by 1,083 HP ProLiant BL280c G6 servers with 8,664 computing cores, the new supercomputer, named “Itasca” by MSI, delivers 97 teraflops of theoretical computing performance.⁽¹⁾ The system delivers three times the aggregate theoretical peak performance of MSI’s other core computing resources.

MSI is celebrating its 25th anniversary as an interdisciplinary research program spanning all the University of Minnesota colleges. Today, MSI supports almost 500 active research groups and more than 4,000 active users across a wide range of disciplines rely on its diverse computational resources.

Among the reasons MSI chose the HP ProLiant BL280c are its scalable quad-core computing performance and memory capabilities. With outstanding dual-processor performance and price per watt, the HP ProLiant BL280c reduces overall data center power consumption while maintaining high performance.

The processing density of the HP ProLiant BL280c is ideal for analyzing massive data sets such as those used for mathematical algorithms and scientific modeling. The HP ProLiant BL280c G6 delivers up to a 190.8 megaflops-per-watt ratio,⁽²⁾ running the TOP500 Linpack Benchmark across a 1056-node system.

The HP supercomputer at MSI features 24 gigabytes of RAM per node, a 40 Gb QDR InfiniBand interconnect and more than 150 terabytes of attached storage. As a result, MSI anticipates improved capacity for running high-performance applications that resolve research challenges.

It’s always cool to see what solutions have been put in place to deliver high performance computing and it’s interesting to note that they are using the BL280c G6. Anything the vendors can do to further the adoption of hpc solutions to meet business or research needs has to be a good thing for the technology and the grid/hpc concepts.

Share and Enjoy

The original Bladewatch site and logo got designed back in 2006 in about an hour, I liked it and we kept that design for a while. As the blog expanded and started to host a little more content, get a bit more exposure, we adopted another series of wordpress templates, we played with the design, the logo and it’s slowly evolved from there.  Over the last few months though we’ve been thinking that the site could do with a bit of a refresh, or reboot if you like, so we’ve deployed the Bladewatch amex, and got some guys to help us out with a new logo and a new site design.

As part of this, the blog will get a new look, a new logo, and I am going to take the opportunity to tidy up the content a bit. It’s easy to start off with everything organized and structured but slowly the content you put up gets hidden amongst other things, so we’ll be re-arranging the documentation, the Dell/HP and IBM pages, the firmware bits to make it a bit more accessible and logical.

I do hope you like the new look when we launch it, we’ve been so pleased with the results and can’t wait to show it to our readers.

Share and Enjoy

I confess that when I first started out, I had no idea how to upgrade system firmware. I started making the offline floppy disks and slowly started to experiment with the online upgrades. I remember having to do a SCSI disk firmware upgrade with 19 floppy disks or something as a Compaq Proliant 2500 was complaining that the disk firmware was too old for my newly upgraded Smart Array 2DH controller running at 4.5.

With this in mind, I’ve made these two documents showing the process to upgrade the firmware online, what screen shots will appear what happens and what to expect or may occur. I’ve named them Array and System. System is about upgrading the system firmware (the bios), Array is about upgrading the firmware on the array controller, the concept for the ILO/ILO2 is the same as are the screens you will see.

• System firmware upgrade Word document and PDF
• Array firmware upgrade Word document and PDF

I’ve updated the HP firmware and documentation pages to host these documents as well.

Share and Enjoy

One of my CIO friends phoned me up and asked,

“Is it me or have we changed the wording on the sales literature from we’re faster and better, to ours is more energy efficient?

I want energy efficient equipment, but that’s not my core concern, my core concerns are:

• Getting through the day
• Perception of delivery and costs  – without the end users/sponsors understanding our operational constraints
• Support costs in terms of the complexity of the infrastructure, the cross charging system having to absorb high costs without necessarily making the end user pay it
• Ongoing discussions about cross charging costs, capex and now opex – you spend how much on on-call staff?
• Issues relating to comparison – everything is compared – they build servers in 19 minutes, why can’t we?
• The ‘you mean we don’t have that already’ scenarios

Energy efficiency is one part, but people forget that I still need to deal with the day to day issues of running a several thousand server estate, cross platform, cross business line and cross geographical lines.”

With that, anyone who phones Tom, or meets him and asks him if he’d like to have a discussion about lowering his data center costs will find they get an uninterested response.

As a vendor, a service provider, I can’t necessarily tell you how to run your IT, or make your costs disappear, however with the right investment combined with process we can start to move away from ‘box selling’ and support, to a service delivery business line. This doesn’t mean that we dissolve who we are, that we loose sight of the technology, it means establishing a dialogue with the user community, putting together with vendors (if required) the business case for investment, ensuring that we have the pre-answers defined, the costs put together and the business case justified.

Share and Enjoy

I was talking with some friends, a project manager, a few windows server guys and a CIO, we had met for drinks at one of the pubs in the city to see how we were all getting on. Anyway, one of the server guys was telling us:

“How ridiculous it is that we’re moving servers from one data center to another, we should be buying new hardware and just moving people on to that, or we could even virtualize it.”

The interesting thing was that the CIO agreed and in essence said we know, but we can’t always say that, there are organizational issues involved, I’ve summarized his responses below.

Why not new hardware or virtualize as part of the move?

For each replacement server I have a minimum of:

• 1 – 2 days build and configuration time from the operating system team
• 1 day for networks/patching
• 1 day for storage allocation, configration and deployment
• 1 day to have the server racked and deployed
• Several days for the application to be loaded, configured and soak tested.
• Are the applications validated on that processor type, on that operating system in that configuration to build version 8.091a?

Let us go back to the original issue. Data center1 is coming to the end of the contract, we therefore need to move to Data Center 2. Now before you say but the age of the hardware, but the energy efficiency and the support cost, remember, in some organizations that:

• There is budget for services, for support, for moving the servers.
• There is no budget to replace your servers.
• The project requirements, the specifications are for a data center move

“To move servers from one location to the other.” I see no mention of hardware refresh or swap out, probably for these kind of reasons:

• There is no budget for capital expenditure – if it is on service, then fill your boots
• The application is coded to a legacy platform and operating system - replacing either would require a re-code or re-testing which is out of scope of the data center move
• The operating system build is not ratified for the hardware available, the build might be ratified, globally certified for HP DL380 G5 or Dell 1950s, but not those new Dell R710 servers that your global purchasing agreement allows you to buy
• The end user does not pay their true support or hardware maintenance cost so the difference is marginal
• The annual internal budget on a departmental basis specifies if they have budget for hardware, they might not have asked for any or wanted to – “leave me alone it works”

We need to be efficient with our infrastructure, with our investment, but it’s a two way street, ‘the business’ need to be asking the right questions, focussing on operational and marginal costs of the infrastructure, not marginal cost to their business line, their application or department. That comes back to the post I did earlier about a user saying, “New laptop, that will come out of my bonus, just rebuild the old one.”

In essence as I have discussed with IT people, end users and vendors, we need to run IT as a business and bring an end to the notion of box selling, support providing and move towards service delivery on the basis of more with less, the maximum benefit, at the lowest operational, marginal and total cost. Your pc died, have a virtual machine or a new one, we are under no circumstances fixing it, because our purchasing agreement means it is cheaper to replace than have a guy spend a day fixing a Pentium II desktop.

Share and Enjoy