The Elephant in the Corner
by Ron Collett | January 31, 2012 | In Chip Industry, Productivity, Project Planning, R&D, Semiconductor Industry, SoCs, design complexity, schedule slip | 1 Comment
Why do so many IC design teams commit to development schedules they know are not possible to meet? I ask this question because it’s such a common occurrence in the semiconductor industry. (Don’t read this article if you never miss schedules.)
Schedule misses are so common as to be an epidemic. It’s as if unrealistic project plans are part of the DNA of the chip industry.
Design teams are loath to complain too much about pie-in-the-sky plans. That’s because they gain little by raising red flags, even though they end up shouldering much of the blame when projects miss schedule. Moreover, complaints are often met with resistance by some of the organization’s stakeholders. It’s just better to play along with the charade, as it increases the likelihood their project plans will get funded.
Once published, such fanciful schedules and resource plans become officially sanctioned propaganda. Just about everybody knows their nonsense, but nobody dares to talk about those big elephants sitting in the corner. At least not until it becomes apparent that the tapeout date will slip—often by months. When it becomes clear that a particular project will badly miss schedule, the organization can collectively and plausibly deny it had any clue that the schedule was unrealistic.
So who’s part of this conspiracy? The genesis is usually in the engineering organization but quickly works its way to marketing and senior management. It starts in engineering because project managers know that submitting resource plans requesting significantly more engineers than management will approve can be career-limiting. Mid-level managers don’t get promoted for saying they can “do more with more.” Yet, in order to finish projects within the time defined by marketing and customers, project managers know full well that additional resources are critical. I’ve personally seen myriad SoC projects staffed with only half the engineers they actually need to finish on time.
Does the conspiracy really start with engineering? I think not. More likely it starts with the leadership of the organization—albeit perhaps tacitly. Of course nobody could ever admit to fostering a culture of self-deception, even if unintentional. Likewise, there will never be acknowledgment, tacit or otherwise, of business strategies whose unintended consequence starves projects of resources—even though it’s obvious projects demand more engineering resources to cope with skyrocketing complexity and ever-tightening market windows. I can’t blame management for trying to keep the lid on spending—it’s business. But failure to make the hard decisions about aligning the product portfolio to match resource capacity is fair game for criticism.
Of course somewhere in this mess sits the unfortunate customer. He’s not savvy to the conspiracy—he never sees the elephant in the corner. He gets a glimpse only when it shows up sitting on his conference room table in the form of the chip vendor’s mea culpa. Of course during this meeting, the vendor parades out the usual specious suspects that caused the delay, but everyone knows what really happened: A gross mismatch between resources, design complexity and schedule constraint. The consequence of the mismatch was an assumption of development productivity that far exceeded what the design team could realistically achieve. Semiconductor companies should get their R&D houses in order, as customers are increasingly on the hunt for elephants.
Originally published: http://www.eetimes.com/electronics-blogs/pop-blog/4235164/The-elephant-in-the-corner
RichMo
1/24/2012 1:05 PM EST
I worked for a major semiconductor company for over 14 years. It seems like at least once a year I would get an urgent summons to a scheduling meeting. In order to meet the customer requirement, they had worked the schedule backwards and determined that my tapeout group would have to work over the Christmas holidays. These meetings would usually be held in June or July so I quickly learned just to say “No problem! If you get your design done on time, I will have a team working.” I never had to fulfill my commitment, because the “elephant” was alive and well!
RCollett
1/25/2012 1:49 PM EST
And who says Christmas doesn’t come in July! Thanks for the comment. Ron
zeeglen
1/24/2012 3:54 PM EST
Right you are, Ron. I remember a manager who told me outright when I said we would need more time that he had to fit the project within the time and monetary budget or it would never get approved. Schedule slip was the usual result, mostly due to unrealistically short development time dreams.
Some upper managers could not comprehend that if it takes a cow nine months to produce a calf, one cannot take nine cows and expect the calf in one month. Yes, the math works, but practical development matters rule.
RCollett
1/25/2012 1:50 PM EST
Having facts data, especially industry data, is the best way to convince skeptics. Thanks for the comment. Ron
tb1
1/24/2012 5:59 PM EST
I actually once worked for a manager who had very accurate schedules, and we actually met every schedule for board and system delivery. I still have a hard time believing it, since I had never seen this behavior before or since.
Unfortunately, the software team was managed more conventionally, and the hardware team was always driving them crazy by delivering hardware on time when they weren’t ready for it.
RCollett
1/25/2012 1:51 PM EST
Maybe that happens more in the systems domain than in semiconductor? Ron
jackOfManyTrades
1/25/2012 3:34 AM EST
When I was an IC designer, I soon learned that the best time to book a holiday (vacation) was around the tapeout date. That way, there was zero chance of the holiday coinciding with the actual tapeout.
any1
1/25/2012 9:30 AM EST
In my experience this deception often starts in marketing. They complain that because engineering is not doing its job fast enough the company is not competitive with its peers and so can’t get enough business to make its sales goals. So then management tells marketing that they can promise better lead times to get those orders and then tells engineering that they have to meet whatever schedule that marketing needs to get the business whether they have the resources to do so or not.
R0ckstar
1/25/2012 6:39 PM EST
That’s basically it. Schedules are generally DICTATED, not requested. A true schedule would be an exercise in impartial forecasting, guessing when some other design team of which you have no stake – will finish. but for many reasons, most of which have already been mentioned, scheduling a project that you have a personal stake in is inherently problematic. If you really want to know how long a project will take, ask the other department, or start a betting pool then watch what the odds tell you. I guarantee it will be dead on every time!
AMSURF
1/25/2012 5:22 PM EST
My favorite excuse for the unrealistic schedule is from the Sales or Marketing person who says, “if we don’t deliver it by this date we will lose the socket and be out of the product’s lifecycle!” My response, “let the race to failure begin!”
zeeglen
1/25/2012 6:45 PM EST
Yup!
Marketing: “Company X is making a fortune with this. We gotta have something similar in six months or we will miss the Market Window.”
Engineering response: “Company X has had a whole year to develop this. We need the same. Why the heck didn’t you guys ask for this six months ago?”
skal_jp
1/25/2012 7:35 PM EST
I remember an argument with my manager:
- We need to tapeout in 1 week
- Sorry, with the machine power and license number we have, I need 2 weeks for running all the backannotation simulation
- Can’t do it in 1 week?
- Nope.
- Then only run part of the sim!
Sim results? There was one pattern with a glitch. Of course in respect this Muphry’s law the glitch was on a pattern I ran after tapeout.
zeeglen
1/25/2012 9:02 PM EST
So who got blamed for not running the whole test?
never mind – I can guess
t.alex
1/26/2012 10:21 AM EST
The customer might be the source of all these in some cases.
RCollett
1/26/2012 3:14 PM EST
No doubt that customers frequently request/demand an accelerated schedule, which is natural — they’re reacting to the changing competitive forces they themselves are facing. Such requests/demands trigger a chain reaction within the chip supplier’s organization — marketing, engineering, program mgmt., senior mgmt., etc. must respond — and the common response is naturally “yes, we can do it.” That’s not surprising either. Everybody wants/needs to please their customers. However, what often happens is that difficult decisions are avoided. Instead, the organization pretends that somehow ten pounds will fit into a five-pound bag. RC
Frank Eory
1/26/2012 5:06 PM EST
In my personal experience, this situation has actually improved a lot over the years. Yes, marketing still dictates a market window that must be hit, and the scheduling works backwards from there — from end of qual, to system validation, to first prototypes, fab cycle time, tapeout, verification time, design time, and to product definition and spec writing before design even gets started.
When that back-to-front scheduling process reveals that design has a negative or unrealistically small positive number of weeks or months to do their job, then the number of choices are very few. If a re-evaluation of the lifetime revenue based on a later market entry still makes business sense, proceed with a new schedule that has the later date. If the resourcing was intentionally lean, add headcount to appropriately resource the project and/or re-define the feature set & scope if a lesser product can still satisfy a segment of the market by meeting the market window.
If none of those business tests pass, then the right answer is to kill the project and put your engineering resources on a different project where you have better odds of making money. Sales will complain that “we’re going to lose that socket,” but the reality is that you have already lost that socket before you even started the project…because you waited too long to start.
RCollett
1/26/2012 5:29 PM EST
Frank, what you describe of course is a rationale approach to a challenging situation. However, my observation — based on the Numetrics customers, which is fairly substantial — is that the situation is actually getting worse. It’s because the enormous competition in the semiconductor industry. We need look only at the industry’s M&A activity during the past few years, as well as the companies forced out of business (e.g. the latest being Trident, which recently filed for bankruptcy). The greater the competition, the greater the need for best-in-class management.
prabhakar_deosthali
1/27/2012 6:01 AM EST
The main reason why schedules are dictated and not requested is that the management thinks that the engineering is always adding a lot of cushion to the timelines to be able to work in a relaxed manner.Mnay of the engineering guys actually do it so that after the squeezing out by the management they get the REAL schedule .
RCollett
1/27/2012 12:57 PM EST
Prabhakar,
So you’re saying that in your experience the engineering organization frequently pads the schedule (timeline)?
Charlie_Edmondson
1/27/2012 3:30 PM EST
There is another side to this problem. A few years ago I was on a project (systems, in this case) where I was the chief hardware designer. My manager was an expert on schedules and budgets, a wiz at anticipating senior managements requests, and an all around good guy. Our hardware team was always on time, on budget, and had a lot of fun.
The SOFTWARE side, unfortunately, was run by a PHB who was always promising things he couldn’t deliver, was way over budget, always behind on the schedule, and continually complaining he didn’t have enough resources, money, people or time.
When the phase of the project I was working on came to a close, the company laid off the entire hardware design team, and spent the next year ‘persuading’ the hardware manager to quit. Why? Because he had OBVIOUSLY been featherbedding his budgets and schedules, and not driving his people hard enough. They promoted the software manager… again!
peinal
1/27/2012 3:34 PM EST
If you think this only occurs in semiconductor design, you’re sadly mistaken. I’ve worked designing HW and SW in military/aero, telecomm, and govt and they all have the same elephant in the room. I once refused to sign a proposal that the govt. required for the preparers. I refused because my initial conservative estimate of 10K hrs was cut to 4K hours on a project that would’ve been 3x more complex than the last one we did (which took 6K hrs). See what I mean?
zeeglen
2/1/2012 6:36 PM EST
A way around this is to multiply your conservative estimate by four times, then when the bean counters divide by four you get the time you really need.
Unfortunately the bean counters eventually catch on to this, so you must multiply your next estimate by 8, next by 16…
WKetel
1/27/2012 8:15 PM EST
My observation has been that sales or marketing will promise anything to get the order, and then engineering and production are blamed when delivery does not happen as promised. This was in the industrial equipment realm, not chip design. Once, a salesman with integrity told us that he would really like the order, but he could not make that delivery. He told us what he could make. Another company claimed no problem with the delivery date, and then wound up being several days later than the supplier that we did not go with. Worse yet, the transducers cost more and were not quite as good. Their catalog got a big red “DO NOT USE” note at that point. I seldom forget a betrayal, and I do take it personally.