After $78 million in losses on the project, the bank announced in 1988 that its trust business was being given to a subsidiary because it could no longer handle the operational requirements. MasterNet quickly became known within the information system industry as a classic case of a system that had fallen far short of expectations. The failure was particularly difficult for Bank of America with its rich history of technological successes. While information system successes have received substantial trade press and academic coverage, system failures have drawn significantly less attention.
Only recently has academia started examining the causes of information system development failure. Clemons has developed a five risk framework for assessing the total risk of a project in an attempt to understand the possible sources of failure. This thesis will examine the MasterNet project using Clemons' five risk framework as a basis of analysis. This analysis will demonstrate that Bank of America did very little to manage the total project risk spanning the five dimensions. Consequently, the project was a likely candidate for failure.
Bank of America’s Master Net Story This section will provide brief coverage of Bank of America’s 1970s history and then proceed through the 1980s with a closer examination of the MasterNet story. A. Tom Clausen's Reign of Neglect On January 1, 1970, Tom Clausen took the reins as Bank of America's president. At the time, Bank of America- tended a stable and profitable retail business that served two and a half million customers. 18 The decentralized retail business proved very easy to -run for the bank's corporate management because an effective set of controls had been established.
Thus Clausen's two predecessors -- S. Clark Beise and Rudolph Peterson -- both looked to diversify and nurture new business. As a result, BofA’s corporate finance and international 'lending grew during these years to become significant enterprises. The move toward making large loans to large corporations worked against A. P Giannini's philosophy that emphasized the "common man," but the new business seemed appropriate for a company that had been successful in traditional commercial banking areas. Clausen took what Beise and Peterson started and pushed the expansion of large corporate and international lending.
This policy fostered several negative effects. First, the overseas operation grew so quickly that the resultant organization was disorganized and lacked the necessary controls for prudent lending. BofA sent inexperienced managers into new countries who, without an understanding of Io- cal conditions, made poor credit decisions. Second, Clausen, in his push for a consistent 10% yearly growth in profits, pressured credit officers into extending lower quality loans. These two conditions significantly weakened BofA’s loan portfolio. Third, the retail operation was ignored and neglected.
While other banks expanded and modernized their retail operations, BofA spent few resources on the retail side-, instead choosing to commit to the new corporate and international lending efforts. The economic downturn of the late 1970s and early 1980s strained BofA’s weak loan portfolio enough to slice into profits. The first quarter of 1981 brought BofA’s announcement of the first decrease in profits in fourteen years. Clausen had taken steps to ensure steady and predict- able profit increases through the 1980s, quite often using accounting manipulations to adjust earnings.
These manipulations were stockpiled as weapons against poor quarters, but by the early 1980s, the stockpile was exhausted. In 1981, Sam Armacost took over as president of Bank of America while Clausen moved on to become head of the World Bank. At the time, BofA’s loan portfolio was falling apart, the bank was large and bureaucratic at the corporate level, and internal systems were strained under increasing pressure. Arrnacost understood the need for stronger operational systems and planned a technological spending program in an attempt to push BofA back into the lead. The phrase that he liked to use was "leap-frogging into the 1990s.
" One area that Armacost emphasized was the trust department because it was mired in a 1965 vintage batch processing system. In 1982, Armacost named Clyde R. Claus executive vice president in charge of the trust department. Claus was given the orders to either fix the outdated department or close the business. BofA’s internal staff had attempted to. develop a modern system in 198 1, but failed after spending $6 million and a full years time. The complete trust-business encompassed three broad areas: 1. 2. 3. Corporate Trust --- the bank acts as registrar and transfer agent for stock and bond issues.
Employee Benefits --- the bank holds and manages assets of private retirement plans. Personal Trust --- the bank helps manage the money for wealthy individuals and estates. The employee benefits area is subdivided into master trust and master custodial accounts. Also, the corporate trust and employee benefit. areas are sometimes referred to together as institutional trusts. The size of the accounts can be as small as a few thousand dollars for individuals, up to billions of dollars for pension funds. Corporate trust is a relatively simple activity requiring: minimal resources.
Employee benefits and personal trust are more complicated because they involve investments in real estate, stocks, bonds, commodities, and other legitimate financial investments. Master trust clients are mostly corporate pension plans that fall within the stricter regulatory reporting requirements of the Employees Retirement Income Security Act of 1975 and related laws. Master custodials provide the same services as the master trusts but management is retained by the client. Master trust and custodials provide such trust services as securities lending, portfolio analysis, performance measurement, benefits disbursement, and record keeping.
For both types, the operation is difficult because of the complexity involved in tracking each account given the varied needs of each client and because of the complexity and dynamics of the government regulations. Consequently, the trust department must provide extensive records of transactions, current positions, and statements of explanation for all actions taken. BofA’s $38 billion of institutional trust assets was split about in half between corporate trusts and employee benefit trusts. Claus quickly discovered that the bank's trust department was necessary to maintain solid client relationships.
Quite often, a client that maintains a large corporate account keeps a. trust account for convenience. After the system development failure in 1981, Claus was reluctant to turn to his internal staff and proceeded to search for an outside. vendor to contract the project. The trust industry held its annual convention in the fall of 1982. At that time two key data processing executives under Claus met Stephen Katz of. Premier Systems. Katz had recently formed Premier after leaving SEI Corp. , an organization that he and Alfred West began. SEI sold a software system based on concepts in Katz’s MBA’s thesis to about 300 banks
through the 1970s, and Katz was entering the same business with Premier. At the convention, Katz met with some BofA officials and began to hammer out a deal for Premier to develop a trust accounting system for BofA. BofA and Katz then proceeded in 1983 to create a consortium of banks in order to share the development costs and risks while participating in the development. The consortium of banks agreed to advance money to Premier to develop a state of the art trust accounting system. The other banks ---Seattle-First National, United Virginia, and Philadelphia National --- were all using an SEI system at the time.
Seattle-First National was a subsidiary of Seafirst Corp. BankAmerica, BofA’s parent holding company, acquired Seafirst early in 1983 when Seafirst began to falter with losses in its substantial holdings of energy loans. Thus, Seattle-First National became a subsidiary of BankAmerica and a sister bank of BofA, although BofA remained a significant part of the total business. B. Development of the MasterNet System Katz and Premier researched the project until March of 1984, when Claus presented Premier's proposal to BofA’s management committee.
The system, called MasterNet, was to consist of a large trust accounting system, called TrustPlus, plus eight smaller systems that augmented the core system. Each system and integrated to provide the full complement of trust automation and would be accessible to remote clients on a real-time basis. BofA’s ultimate goal was to sell the trust accounting services of the system to small and mid-size banks. MasterNet's initial budget was 420 million and the completion date was set at December 31, 1984. Katz's design engineers at Premier were to work with BofA’s systems engineering and trust department to develop MasterNet.
The design and implementation of the computer system followed a reasonably standard plan. A committee of all departments affected by the- new system, including those from the consortium banks, met monthly to define requirements. BofA’s data-processing staff met weekly with Premier's designers to discuss progress and needs. Also, expert users were assigned from all banks to become involved with the design process and provide continuity from design to implementation. Data processing executives felt that the degree of camaraderie and cooperation were exceptional throughout the design process.
Also, BofA took significant steps to ease conversion to the new system. A comprehensive training program was implemented that as- signed user/trainers to develop courses including videotape, classroom, and hands-on terminals. The material was so well-de- signed that BofA earned a $1. 5 million grant from a California state' program that rewards companies for committing resources to employee retraining. Later, these knowledgeable trainers served as code testers and certifiers. Though the initial December 31, 1984 deadline passed, Claus was not worried because of the progress in the system's development.
Meanwhile, BofA was restructuring the. organization of the bank under the guidance of Armacost's personally hired management consultant, Ichak Adizes. Armacost dreamed of a united team of executives supporting his vision of the bank and hired Adizes to orchestrate the effort. One outcome of Adizes'work was the formation of BASE --- BankAmerica Systems Engineering --- a consolidation of the assortment of systems engineering departments at BofA. BASE was headed by Max Hopper of American Airlines' SABRE fame and would be responsible for the management, development, and application of technology.
BASE also spearheaded a $5 billion, five year technological spending program that the bank announced at the same time. Clausen's neglect of systems during his reign al- lowed a collection of over sixty networks to develop worldwide at BofA. Although the systems were reliable, the whole situation need- ed consolidation in order- to allow orderly expansion and improvement. Organizing BASE was a first step toward this goal by bringing together the various groups responsible for the bank's domestic and international computer and telecommunications operations.
BofA expected that BASE would not only improve operations but sharpen the bank's competitive edge as well. As a result of the formation of BASE, Claus lost authority over his portion of the systems engineering department. He also lost the securities clearing operation due to other restructuring activities. Consequently, he found it increasingly difficult to get these two groups and his trust department to work together. Despite this, MasterNet development continued through 1985 and into 1986 until March . when Claus thought that the system was ready to be publicly announced.
In May, BofA staged a lavish two-day demonstration of the system. Many of the bank's most important corporate clients attended the $75,000 party that touted the "industry's most sophisticated technology for handling trust accounts. Claus was positive about the presentation and felt that clients were impressed by the system's advanced technology. Claus apparently believed that the system was ready for production use, while other executives felt the show was staged merely to appease anxious customers. During 1986, Claus' department began to prepare for the conversion to MasterNet.
The first step called for the movement of the $38 billion worth of. institutional trust customers first. The smaller consumer. division accounts would be converted later. The department made repeated attempts in late 1986, but were continually stopped by technical problems. The most glaring problems included poor response time and days-long system crashes. Despite the problems, Claus’ department pressed on, putting in long hours in continuous attempts to get the system up and running. The enormity of the con- version task made it particularly difficult.
Every single client asset had to be classified into one of the approximately 128 asset types. Personal notes kept by trust employees had to be gathered, re- viewed, updated, and loaded into the system, entirely separate from the financial data. Meanwhile, Bank of America as a whole was having significant problems. 1985 and 1986 brought the bank losses of $337 million and $518 million respectively. Since Armacost had become president, problem loans had been uncovered at an increasing rate. Also, news of the bank's losses and bad loans fueled rumors that BofA was in danger of failing.
These rumors caused confusion in California where consumers began to close their accounts, causing a loss of more than $2 billion out of $44 billion of domestic deposits. These problems forced BofA’s board of directors to oust Armacost and replace him and his vision of technology with the returning Tom Clausen. In October of 1986, Clausen regained control and immediately began cutting costs and selling assets in an effort to restore profitability. C. MasterNet’s Failure and Postmortem In early 1987, the system stability had improved enough to consider a serious conversion effort.
A date of March 2 was set as the conversion deadline. 'As the systems engineering staff of sixteen rushed to complete the preparations, half of the group was pulled off of the project. In an effort to offset losses, BofA sold its consumer trust business to Wells Fargo for $100 million. The proceeds could be booked in the first quarter if the deal was closed by March 3 1, so the half of the staff that was pulled was assigned to transfer the accounts to the SF-1-based Wells Fargo system. The remaining staff worked continuously up to March 2 and all of the institutional accounts were transferred by the deadline.
Six days later, the first of over a- dozen Prime disk drives failed. The staff spent a weekend downloading back-up data and the rest of the month dealing with the remaining failures. It turned out that the faults inherent in the Prime disk drives didn't show up in the first months of testing. Eventually, 21 of the 24 disk drives were replaced. BofA decided in April of 1986 to move BASE to Concord, a suburb about half an hour cast of San Francisco, in an attempt to tap the synergies of a large group of technologists and hopefully create better programs faster.
It was an opportunity for the bank to replace antiquated technology with cutting edge technology and consolidate sixteen separate groups spread across San Francisco. The new technology center was also intended to attract recent college graduates. Several key employees quit and morale sank as the pressure and stress of the last few months combined with the move announcement pushed them over the edge. Similar difficulties had arisen in the securities clearing operation based in Los Angeles the previous month. This group was responsible for the tracking and reconciliation of the purchase or sale of stocks or other securities.
In March, BofA announced that this group was to be moved to San Francisco. The reaction here, similar to that of the data-processing group, was employee dissatisfaction and defection. The bank eventually put the decision on hold for three months, but the damage to morale had been done. In order to fill the gap in Los Angeles, high-paid consult- ants were brought in to complete the work. Meanwhile, improvements to MasterNet were being made and many of the problems of the previous year were cleared up. Unfortunately, new problems surfaced-for example, two new types of system halts were discovered in spring.
Also, because the processors were fully utilized an additional two units were added to the current three. The hardware and software problems created an operational backlog that delayed processes such as monthly statement generation. Some accounts received monthly statements as much as two months late. Also, -the- backlog made it difficult for the trust department to maintain current data. In an effort to keep updated, fund managers were forced to get current information on portfolios directly from investment managers instead of through MasterNet.
Many began to consider switching to other organizations to handle their trust business. In July of 1987, BofA announced that it was reserving $23 million related to losses due to MasterNet problems. This reserve covered costs related to the hiring of consultants and accountants needed to clean up the damage, expected loss of fees, compensation claims for securities delivered late, and potential losses due to transaction discrepancies. While BofA publicly assured others that the problems would be solved, it was quietly seeking a buyer for the institutional trust business.
In October of 1987, both Claus and Mertes resigned in wake of the problems. The bank immediately assembled a seven member team to handle the MasterNet problem. Michael Simmons, formerly in charge of computers and telecommunications at Fidelity Investments in Boston, was hired in July of 1988 to replace Me-rtes as head of BASE. The stock market crash of late October exacerbated the problems with the transaction processing and statement generation portion of the system. Software industry sources said none of the other banks involved in MasterNet had become as deeply enmeshed with MasterNet as BofA.
Philadelphia National Bank dropped out of the consortium two years before. United Virginia Bank used MasterNet only for its custodial trust business, which is less complicated than master trust services, and Seattle-First National appeared to have delayed adopting MasterNet in its employee benefits business. In January of 1988, BofA announced that an additional $35 million would be added to the current $23 million reserve for MasterNet malfunctions. In comparison, BofA earned $60 million in the fourth quarter of 1987. Business began to evaporate as clients pulled their accounts from BofA.
The number of accounts dropped from 800 to about 700 and total institutional assets declined to $34 billion from $38 billion. A few days after this first announcement, BofA announced that 95% of its institutional trust services clients would be shifted to BankAmerica's Seattle-First National subsidiary. The remaining 29 clients, representing BofA’s largest and most complex accounts, would be given outright to State Street Bank and Trust Co. In May of 1988, BofA completed the conversion of its trust ac- count processing to a service bureau system running at the SEI data center while Seattle-First National handled day-to-day trust operations
for its sister subsidiary. Thus, four years after the start of the project, not only was the system a major failure, but the whole trust business was lost. A total of $80 million was spent on the "stillborn" system and BofA received substantial press coverage of the debacle. The LA. Times said in a front page story that BofA slipped in its "bid to leap into 1990s technology," while ComputerWorld, an information systems trade newspaper, called MasterNet an "$80 Million MIS Disaster. " These words were especially harsh given BofA’s technological leadership throughout most of its history.
At a time when BofA was struggling to regain profitability, the bad press helped convince the world that BofA really did have significant internal problems. At the same time, BofA was also getting press coverage for its loan losses and attempts to stay afloat as one of the largest banks in the country. MasterNet is now considered a classic example of an information technology failure. IV. A Risk Assessment of MasterNet This section will assess the risks that Bank of America was exposed to with the MasterNet project.
First, Kemerer and Sosa's work and Clemons' follow-up work in the area of risk management will be re- viewed. Then, the MasterNet project will be analyzed using Clemons' five-point risk assessment structure. A. Review of Previous Works In their 1991 paper "Systems Development Risks in Strategic Information Systems," Kemerey and Sosa examine strategic information systems (SISs) that have fallen short of expectations. In contrast, current writings use such. well-known successes as American Hospital Supply’s ASAP order entry system as encoura ment for the use of SISs.
Kemerer and Sosa chose to examine failed SISs in an attempt to identify the barriers that prevent successful SIS development. Their thesis states that "there exist significant systems development challenger, that present risks or even barriers to some org4nizations'attempts to use IT strategically, and that executives and systems developers who are considering an SIS development must plan carefully to avoid these. pitfalls and increase the likelihood of a successful SIS. " Kemerer and Sosa illustrated their position by collecting a broad array of SIS failures and categorizing the types of problems that plagued them.
These problems were divided. into the- three phases of the systems development life-cycle model: definition, implementation, and maintenance. Kemerer and Sosa then developed a matrix that corresponds the particular pitfalls with the firm's relative position along three dimensions: monetary resources, technological sophistication, and organizational flexibility The purpose of the matrix is to allow firms to identify their most likely pitfalls according to the characteristics of their organization and then adjust their. plans to balance risks as they desire.
Clemons' 199l paper "Evaluation of Strategic Investments in Information Technology" took the evaluation process one step further and developed seven lessons that can be applied to the evaluation of a firm's investment decision in SISs. Clemons' third lesson --- it is necessary to balance many forms of risk --- lists five basic components of risk which must be managed and traded off depending on the firm's Comfort level. These five risks are classified as financial, technical, project, functional, and systemic. B. The Five-Risk Analysis This section is devoted to assessing the MasterNet project along Clemons' five-risk framework.
Five subsections follow, each devoted to a particular risk. 1. Financial Risk The first of Clemons' five -risks is financial risk. Clemons defines excessive financial -risk as unacceptable financial exposure or costs that are out of line with expected benefits. The examination of Master- Net's financial risk will concentrate mainly on. Bank of America's weak overall financial condition and the significant downside financial exposure of MasterNet. During the MasterNet project period, BofA’s finances can best be described as weak. In the early 1980s, Bank of America experienced a severe downturn in its financial condition.
In 1982, BankAmerica (the parent holding company whose most substantial holding is BofA) posted a $457 million profit on $4. 2 billion of revenues. This is a mere 2. 2% increase from the year before. At this time, the discovery of bad loans was accelerating and its true exposure to bad credit was just being realized. BankAmerica posted even lower profits in 1983 and 1984, until it posted a. $337 million loss in 1985. The organization bottomed out in 1987 with close to a $1 billion loss. Mentioned earlier, the bank set out on an aggressive $5 billion, five year systems spending plan in 1985.
With revenues of $5. 3 billion in the same year, Armacost and Hopper planned on spending nearly 20% of revenues on systems maintenance and improvement alone. The $1 billion per year figure is over 25% higher than the spending in 1984 of $780 million. This spending was concentrated in BASE, a group consisting of less than 10% of the bank's employees. Relative to $5 billion, the $20 million spent on MasterNet is a . 4 % portion. While Armacost committed huge sums to technology, he was also slashing costs in other parts of the bank in an attempt to streamline.
Armacost withdrew services, closed about 200 branches since 1981, and jettisoned entire lines of business including the FinanceAmerica subsidiary, a national consumer loan network. Armacost was taking BofA in an entirely new direction, shifting expenses away from the bloated retail, business and corporate bureaucracy into systems spending. Other banks in the master trust industry were also spending significant money on their systems. From 1983 to 1987, First Pennsylvania Bank spent $8 million to develop a new master trust system. The bank had $31 billion in managed master trust and custodial as sets as of mid-1987.
Also, Banker's Trust planned on spending $10 million in the late 1980s to update its master trust capabilities. It had $118 billion in master trust and custodial assets as the third largest master trust bank in the country. Banker's had a 1977 vintage system that worked efficiently --- the money was budgeted for incremental improvements, such as the ability to handle more sophisticated securities. While the basic MasterNet project was budgeted for $20 million, the potential loss due to system failure could be higher due to other, auxiliary factors.
These potential factors include: Completion of Work – A system failure such as MasterNet's forces the bank to perform the intended services in an ad hoc manner, usually via manual labor. For example, in 1983, BofA’s investment Securities Division (BISD) installed a new computer system to handle the volume of transactions flooding the operation. The new system was started. up while the old system was shut down. Unfortunately, the new system worked improperly, misrecording thousands of transactions. Teams of auditors were brought in to sort through every transaction by hand.
In this case, a $20 million reserve was set up to cover losses and audit expenses. Inaccurate Transaction Recording - When a bank cannot accurately record transactions, it must resort to "blind settling"-the practice of accepting-the transaction terms that the counter-party recorded. If the comparison of BofA’s records and the counter-party's records indicate a discrepancy, then the counter-party is normally not paid until verification. When blind settling, the counter-party is automatically paid and the- exception is verified manually. BofA had such
a huge backlog of exceptions that it would be impossible to retrieve all overpayments due to the extreme delay. Inaccurate Asset Tracking -The MasterNet system was designed to track the asset position of its accounts. Since-BofA managed $38 billion institutional dollars, the exposure to mistakes was significant. The personal trust business was never transferred to the system, so it was not at risk. Loss of Managed Assets - Before the institutional trust business was given away, the total managed assets had slid from $38 billion to $34 billion. This represents a 10% decrease and an approximate revenue loss of 10%.
Loss of the Business-Obviously, the giveaway of the institutional business to Seattle-First National and State Street represents a total loss of all future cashflows. BofA was fortunate to sell the personal trust business and obtain compensation. Loss of Peripheral Business-The press coverage of the MasterNet project damaged BofA’s reputation. While the effect on BofA’s development of new business was unknown, the event certainly had a negative effect. Litigation and Fines-In the regulated trust business, banks can be fined by the government for not adhering to the regulations. Litigation costs include lawyers and court costs.
In retrospect, BofA spent an additional $58 million over the original $20 million in direct costs cleaning up the MasterNet mess. The basic financial benefit of the MasterNet project is not so much the generation of new cashflows but the continuation of current cashfiows. In 1987, BofA made nearly $100 million on its trust business despite the MasterNet problems. MasterNet was primarily intended to be a “catch-up” project with the highest priority of continuing the old system's operation, albeit more efficiently. A secondary goal of the MasterNet system was to add enough functionality to attract new clients.
This new business could consist of additional trust accounting or the sale of the trust accounting services. Management thought the real value of the project rested in the new technology. If by end of the 1980s BofA had a 1965 vintage system, it is clear that the bank would have little strategic edge in this highly competitive arena. Because of the risks involved, the bank did attempt to reduce the financial exposure. The consortium was formed primarily to reduce exposure by spreading costs among the group, but the inclusion of Seattle-First National made this diversification less effective.
Seattle-First National, like BofA, was a subsidiary of BankAmerica (the parent holding company). Therefore, the financial risk was not really reduced for BankAmerica, because the finances of its BofA and Seattle- First National subsidiaries are consolidated into its own financial statements. Keeping the banks involved with the project was another problem. In 1985, Philadelphia NationalBank dropped out, leaving just three banks remaining. Bank of America found it necessary to undertake the MasterNet project. In assessing BofA financial risk with MasterNet, two items in particular must be considered.
First, the overall financial health of the bank was weak and deteriorating. Second, the downside costs of failure were significant given that MasterNet handled billions of dollars of client assets. 2. Technical Risk Clemons defines the second risk, technical risk, as the possibility that the project just cannot be accomplished because the supporting technology is not available. This section will take a broad view of this definition and include not just the chosen technology, but the process of converting from- the old to the new. In this context, technology refers to the basic processing platform, the surrounding.
telecommunications network, and the necessary software. MasterNet was an aggressive attempt to provide a full complement of trust automation. Most of this functionality was directed at the master trust and custodial trust business, both of which are significantly more complicated than the corporate trust business. The core trust accounting system turned out to be the most difficult to implement. In fact, six of the eight subsystems were successfully implemented and used within the bank. MasterNet planned functionality was grand: an international network, full customer access, and on-line reporting.
, Initial designs called for the use of a single eight megabyte, one MIP Prime processor. As database and functionality requirements, grew through the development, the system grew to three sixteen megabyte, eight MIP Prime processors on the conversion date in March of 1987.
This represents a 24-times increase in computing power. Two of the processors were used as front-end devices, while a single processor connected to thirteen 600 megabyte disk drives served as the back-end, This configuration required development of various new components: a special data communications channel between CPUS, changes to Prime's PRIMOS operating system, changes to Premier's NEXIJS database, and special RAM disk storage for paging.
The development of these components stressed the design groups at BofA, Premier, and Prime. As the operational backlog grew through 1987, two more processors were added to expedite statement generation. The decision to use Prime hardware 'was driven by Steven Katz of Premier. BofA succumbed to Katz despite the fact that BofA had been an IBM house since, 1955. A