Friday, July 16, 2010

$1.2B still get monkeys....

DBS paid $1.2B to IBM in an outsourcing contract for 10 years in 2002. As part of the deal, 500 DBS employees who formed the IT team were hired by IBM[Link] in one of the largest outsourcing deals ever in Singapore. Outsourcing became fashionable about 10 years ago and there was a rush by companies to outsource their inhouse non-core activities to other companies. You wonder how IBM which simply took over the staff from DBS can do a better cheaper job and still turn a profit for itself doing essentially the same thing DBS was doing inhouse. I'm not here to condemn IBM or DBS for the recent debacle - there's enough of that by other bloggers and netizens already. I'm familiar with IT systems and can understand how small mistakes can lead to severe consequences. In this case, it was reported that the IBM maintenance team was given a set of just outdated (outdated by a few days?) procedure by a regional team and that led to the shutdown. Quite unlucky....and you can feel sorry for the engineers who simply followed the wrong instructions given to them leading to the mess. At the end of the day, it could boil down to one person looking up the wrong instructions....this type of human error cannot be avoided unless other checks and processes are in place e.g. someone else has to sign off the instructions and so on. But sometimes systems are so complex, you cannot tell that the set of instructions are so critical and there is just no way to prevent it unless you put in checks all over the place which can be quite inefficient. So even IBM makes mistakes just like everyone else....but you notice how quickly the blame shifts from DBS to IBM and because it is IBM a big established brand name. Suppose the same 500 staff from DBS formed a company called XYZ and DBS gave the contract to them and the same thing happened, it is harder for DBS to offload the blame to a no-branded company. If DBS had not outsourced to IBM, it would have to take all the blame on its own.

When I started work many years ago, I was tasked to buy a piece of computing equipment. I looked around and the best deal was from a lesser known (no brand) company which sold the equipment for half the price of similar HP and IBM equipment. It did some checks and found that the equipment was really value for money but was given advice by a more experienced colleague not to buy it. The logic is like this - if it is IBM and it breaks down, the management will blame IBM.... but if I buy an unknown brand equipment and it breaks down, they will blame me for choosing the no brand one even though I did a proper assessment. I won't tell you what I eventually bought but in most companies, people will buy the branded equipment because they rather use the company's money to buy something more expensive than to take on the risk and responsibility. The difference in this type of decisions is the result of company culture and the willingness of people to take responsibility and risk ultimately determines of a company can compete and survive. Many companies start off small and lean then before they find success. When they become richer, they can lose the 'can do' spirit and the management starts hiding behind consultants who are hired for expertise they are suppose to possess so that they can blame the consultant for bad decisions...when responsibility and blame can be outsourced by paying money ...why not.

56 comments:

Louis Tan said...

You wonder how IBM which simply took over the staff from DBS can do a better cheaper job and still turn a profit for itself doing essentially the same thing DBS was doing inhouse.

i'm surprised you seem to be unfamiliar with the concept of economies of scale?

Investment said...

The glitch is really regrettable!

skeptic said...

Moral of the story? Invest in IBM stock because of their economic moat.

Lucky Tan said...

Louis Tan,

For manufacturing there is plenty to gain from economies of scale. However for service industry, it is much less. Anyway, with 500 IT professional equivalent to the size of IT company the DBS IT team does not suffer from disadvantage of small size.

Anonymous said...

Its funny when one link the accountability to outsourcing. Isnt't that is what is happening all around us? DBS is not alone, the SAF outsourced food and general maintenance, and now the running of firing ranges. If I were the SAF might as well employ mecenaries. The police outsource to AETOS, the ROV to LTA patrols. Im confused and maybe wrong here but thats the gist.

SNA said...

It is all about GDP. The cost of outsourcing to IBM adds to the GDP, which otherwise would have disappear under wage expenses if it is done in-house. This is just one of the many ways of cooking up GDP numbers.

xl said...

Larger companies are generally staffed with cowards. & 1 reason why they join larger companies is because they need 'something big' to bloster something they are lacking in personality.

Is same as pap. Grow bigger & lulled into complacency. Most ministers are cowardice in their policy making & behavior. Too much to lose so better play safe safe as per status quo.

How to play safe safe? eg. GLCs suck own citizens than make bulk of their revenue from overseaas.

Keep to the status quo aka 'Maginot Line' mentality would doom spore 1 day.

Anonymous said...

In other words, another example of "too big to fail" hahaha

recruit ong

Anonymous said...

Anyone got read today Today newspaper?

There is a landmark court case whereby the blog owner got sued for defamatory remarks made by blog commenter, rather than the commenters themselves.

The government finally come up with a way to deal with social-political blogs once and for all.

The implications of the outcome for this case is that if blog owner is the one being sued for comments made by others, then they can easily used the Internet Brigade Corps to post racist and religious comments and then come after the blog owner, hereby preventing the greatest threat to them - alternative views that can be found only online

I'm afraid that we cannot change the government.

Anonymous said...

DBS is a bank. Its core business is banking. So IT functions, not being core business, will be outsourced, just like what other companies do for non core activities.

IBM core business is IT, and they not only got from DBS but also from other companies the IT business. Hence it is hoped that being their core business, IBM will do a better and cost effective job for IT functions than if the companies did it themselves. Or so their clients thought.

But to me, despite such IT glitches that DBS recently had, I don't think their business will be affected. Simply because they have the largest ATM network and hence is also the preferred bank of employers for staff salary crediting. So what choice do the account holders have?

Just like the people have no better chocie even when MRT is overcrowded. Or if HDB flats are very expensive.

And to top it all, the people have no better choice or no choice when it comes to voting.

It all boils down to what choice do you have. And if it is Hobson's choice, you will have to choose the lesser of 2 evils.

Lucky Tan said...

anon 10:14,

For many things like cleaners, canteen and building maintenance etc it make sense to outsource. For small companies they even outsource their accounting etc because they need like 1.5 people to do it and they save cost outsourcing. In DBS's case it is debatable because of the size of the IT team and the domain knowledge for banking sector software. It is this domain knowledge that is hard - IT skills like software is common place. To have an IT team that has this domain knowledge ..it is questionable whether to outsource. I don't think the other banks (can someone confirm?) outsource their sizeable IT operations. DBS ride on the outsourcing wave when it was fashionable like many companies and some of these companies outsourced activities that shouldn't be outsourced.

Lucky Tan said...

anon 10:14,

Took this from the media report:

Rival banks OCBC and United Overseas Bank have taken a different tack and run most of their IT functions in-house. Bankers there say that while outsourcing brings benefits like cost savings, they do not farm out key IT functions given the risk of losing control over vital operations and compromising confidential business information.

'We outsource selectively, such as getting the vendor to do coding projects or in application development,' an OCBC banker said. 'But not outsourcing our key IT infrastructure.' He said if they outsourced, they would manage the vendor 'tightly'. OCBC manages its IT operations through its own subsidiary, e2power, with its staff seconded to help out.

A UOB banker said the bank outsources in selected areas, for instance in statements printed by Singapore Post. But these are not 'mission-critical' activities like the running of its data centre, which is managed internally.

Anonymous said...

When choosing which IT systems to go for, just select IBM.
--- this saying was popularised way back in the 1970s, and is still in practise today after 40 years!

Back in the early to mid-1990s, MOF and IRAS also outsourced responsibility for e-taxing to Accenture. Those in IT industry would remember that this project was not a happy one --- grinding, soul-sapping and acrimonious, with plenty of finger-pointing. When IRAS finally announced to Singaporeans about e-taxing and encouraging everyone to file taxes online, the system was still so buggy and so slow (after spending S$1Billion) that the then Finance Minister personally picked up the phone and called Accenture's partners, telling them he would sue them until they became bankrupt. Accenture finally tracked down a founding partner, flew him into Singapore and he was able to decipher the actual machine code of the IBM mainframe to solve the problem.

Nowadays it is much easier to select among various big-name and famous consultancies and get them to compete in tenders and provide prototyping. The only thing you need is plenty of bullets and bombs to sue in case anything goes wrong, which it will. Unfortunately, ALL big-name consultancies mainly hire lowest-cost ah-nehs and chinks to implement IT-related solutions so the standard and outputs from all big-name consultancies are all about the same.

Anonymous said...

from what i know hardware maintenance is always done by supplier, in this case IBM. no inhouse IT staff will mess around with hardware on a production system.

Anonymous said...

What is there to say? DBS has always been and is the worst bank in Singapore. And do you all know why? Because "someone" who has zero experience uses it as a proxy to indulge in their own fantasy and delusion of being a banker and so on... cue GIC, Temasick.

Colin said...

Document control is the process used to ensure that only the right and correct documents are used. Its part of any ISO 9001 organisations processes. Should never happen to an IT company as normally these are controlled online.

Outsourcing is a trend although I have yet to see definitive proof that it is more effective in technical type areas.

runroad said...

"No one ever got fired for buying IBM” was the popular, cynical, axiom for grey corporate drones only concerned with protecting their own rear ends in the late 70s. That was the reason why competing mainframe makes like Amdahl, ICL, Fujitsu et al with better, cheaper products found it impossible to break through IBM's stranglehold.

The 90s saw 'IBM' replaced with 'Microsoft' and in the present recession, 'gold'. As Lucky points out, picking the industry leader because it's the safe option allows you to point the finger instead of accepting personal responsibility should anything bad happen. It's the perfect, irresistable kiasu choice. And naturally the collapse of DBS' IT systems is the delicious middle-fingered come-uppance for the ultimate kiasu nation. Couldn't have happened to a nicer bunch.

Anonymous said...

'Its funny when one link the accountability to outsourcing. Isnt't that is what is happening all around us? DBS is not alone, the SAF outsourced food and general maintenance,'

If SAF lost the war against foreign aggressor invading singapore, do we blame it on outsourcing of these services?

Anonymous said...

The compromises of national security and hence safeties of the state and its' inhabitants are obviously objectionable.

Changi Airport ranks amongst the most important installation and yet the securities of it are much handle by foreigner staff from Malaysia. Many of the security personnels deployed there are foreigners.

Me am inclined to believe that Jurong Island would be similarly 'secured' and 'protected' by foreigners.

Instead of having our lands, people and installations protected, me just feel rightly or wrongly, the compromises endangers our wellbeings.

patriot

Anonymous said...

Mr Lucy Tan,

You have been highly productive in recent weeks; one article after another.

Despite such high productivity, the quality of your writing remains.

Many readers and I benefit from your wisdom. I want to express my thanks to you.

PLEASE KEEP THEM COMING...

monkeys in white said...

To anon 16:44,
You are right about Lucky.

And like many Singaporeans posting on his blog, Lucky has a v active mind.

It is good for the country but will be hell of alot better if many can replace the 82 monkeys with one dominant male in parliament.

Anonymous said...

Those who are in IT industry will know that outsourcing is one of the ways of "racing to the bottom", just like why companies are relocating to low cost countries for production.
Outsourcing is beneficial to those IT Big boys as they can offer breadth and depth, and they increasingly are playing the role of main contractor and re-source again to secondary vendor. With so many hands involved, there is definitely a drop in product and service standards, but now we have a tier-service standard, as long as the VIP users are not affected, then it is ok.

The sad state of affairs arise because IT is the classic example of more sellers than buyers. If you flood the workplace market with a lot of cheap supply, "race to the bottom" will happen and the buyers will gleefully get the cheapest supply to derive maximum benefits. Such short-term view will definitely haunt the buyer in future, as again with a lot of changing of hands (due to resignation, re-appointment of secondary vendor etc), it will result in documentation not updated or contain errors, and thus mistakes will happen, more so for those complex systems.

Anonymous said...

Pay peanuts u get monkeys. Pay more peanuts u get more greedy monkeys. That is what they Singapore political situation has demonstrated.

Anonymous said...

it has instead created a huge dent on DBS bank's reputation. just like SIA that their plane will not crashed.
outsource selectively is key word and DBS has not learned their lessons well.
it will happened again.

Anonymous said...

Mr Lucky

//
but in most companies, people will buy the branded equipment because
//
the branded company will be around to do servicing/repairs?

Have you heard of "Total Cost of Ownership"?
Given your limited experience and knowledge, no doubt sticking with IBM would be the correct choice.

Btw, I dun agree with outsourcing (completely unrelated to your experience). Crap banks view IT as non-core. The Goldmans and JPM invests heavily in IT (very much "core") and makes shitloads of $$$ from IT (eg HFT, etc). $550M is like a rounding error ... 93cents per share (Gldman trades ard $140?)

Anonymous said...

tLucky Tan 16/7/10 11:20 wrote: "I don't think the other banks (can someone confirm?) outsource their sizeable IT operations."

Agree that around 10 years ago, outsourcing was a management trend that many bought like the latest trendy fashion only to regret later.

Citibank outsourced its IT functions in the early 2000's. By 2003 to 2004, it backtracked and started back-sourcing selected IT functions, e.g. Private Banking.

ABN AMRO (now bought over by RBS) in the 2003 outsourced its IT functions to EDS (now bought over by HP). By 2005 to 2006, it started back-sourcing selected IT functions too.

Even GIC (yes, the one that lost our nation's wealth) outsourced most of its IT development. Suffice to say that it bought an outsourcing vendor's marketing bullshit about having strong finance domain knowledge hook-line-and-sinker.

Anonymous at 16/7/10 11:37 wrote: "Unfortunately, ALL big-name consultancies mainly hire lowest-cost ah-nehs and chinks to implement IT-related solutions so the standard and outputs from all big-name consultancies are all about the same."

Ha, ha, so true. I'm so glad that I have left the industry. Agree also with the observations by Anonymous at 16/7/10 20:32.

Anonymous said...

Being in IT industry, I would say outsourcing plays a big part as IT Jobs is getting getting cheaper and commoditize to the extent that you can hire IT Technical Assistant for just $1200 just $100 shy from Pest Exterminator.

I don't blame outsourced staff don't feel affiliation to the company task to support. There are many instance where by disaster recovery plan and business continuity plan will shelf by higher management as they deem IT as cost centre rather than a necessity to drive business. They would rather channel funds to sales & marketing that will bring in the bacon.

Anonymous said...

buy ambien can i take 2 ambien cr 12.5 - ambien pregnancy class b

Anonymous said...

buy xanax 1mg alprazolam 0.5 mg and alcohol - xanax xr 0.5 mg tablet

Anonymous said...

diazepam 5mg diazepam for sale no prescription usa - cheapest diazepam online no prescription

Anonymous said...

buying xanax xanax withdrawal regimen - how to order xanax online

Anonymous said...

valium 10mg valium drug ingredients - online purchase of valium

Anonymous said...

zolpidem without a prescription zolpidem tartrate 10 mg erowid - price of zolpidem tartrate

Anonymous said...

diazepam online buy valium usa cheap - buying diazepam online legal

Anonymous said...

online xanax no prescription xanax dose for recreational use - has bought xanax online

Anonymous said...

diazepam is valium diazepam 2mg dentist - diazepam 5 mg prescribed for

Anonymous said...

buy alprazolam online no prescription buy xanax canada no prescription - should buy xanax online

Anonymous said...

diazepam 10 mg diazepam side effects memory - valium diazepam for dogs

Anonymous said...

generic lorazepam ativan dosage effects - ativan etoh withdrawal

Anonymous said...

alprazolam without prescription xanax pies - up your xanax dosage

Anonymous said...

buy ativan online ativan cost - ativan 321 m

Anonymous said...

buy diazepam online diazepam dosage bnf - diazepam online buy

Anonymous said...

generic lorazepam does ativan and xanax show up the same - ativan withdrawal and dizziness

Anonymous said...

diazepam no prescription needed buy diazepam safely - diazepam online rezeptfrei

Anonymous said...

xanax cheap much xanax pills - order xanax forum

Anonymous said...

online ambien ambien 10 mg looks like - how much does generic ambien cost at walmart

Anonymous said...

buy soma getting high carisoprodol - online coupons for soma intimates

Anonymous said...

soma generic soma drug abuse signs - soma what drug class

Anonymous said...

buy ambien online buy ambien cr online no prescription - ambien withdrawal period

Anonymous said...

somas online what is soma drug classification - buy soma echeck

Anonymous said...

buy valium online valium 2mg online - buy real valium online

Anonymous said...

buy valium online is it legal to buy valium online - valium online no prescription uk

Anonymous said...

buy soma buy soma/ watson brand com - soma drug in book

Anonymous said...

buy valium online order valium from thailand - valium pill code

Anonymous said...

buy soma online soma drug screen - generic soma online

Anonymous said...

cheap ambien online blue ambien pill mg - best place buy ambien online