Numetrics » Productivity

Why They Benchmark Productivity

Ron Collett — Sat, 26 May 2012 00:15:44 +0000

Why do semiconductor organizations benchmark product development productivity? Two reasons. The first is obvious—to determine how their product development competitiveness compares against the industry. R&D prowess is a matter of long-term survival. Second, measuring their productivity enables reliable forecasting of engineering headcount requirements when planning new IC projects. Accurate forecasts equate to both on-time schedule performance and high schedule predictability. It’s a matter of competitive advantage.

Creating consistently reliable project plans requires a solid grasp of the R&D organization’s development productivity. That’s because productivity dictates how many engineers a project needs to finish on time. Too few engineers and the project slips schedule—a common occurrence. Organizations measuring their productivity calculate exactly how many engineers projects need.

Quantifying productivity also enables organizations to create better product roadmaps. R&D managers calculate how many chips (across the target design complexity range) the organization can develop annually.

Organizations whose productivity is higher than the industry quickly find they’re able to allocate fewer engineers per project than competitors. That yields a big competitive advantage—lower headcount translates to lower development cost. High productivity also means the R&D organization has additional resources that otherwise wouldn’t be available.

They use those “additional” engineers in a number of ways. First, the organization typically develops more products than competitors. Second, it often develops products that have higher functionality/performance and therefore more value-add, which brings higher profit margins. Third, they frequently “overstaff” critical projects to reduce development time and time-to-market. Larger project teams almost always exhibit higher development throughput, or output per unit of time.

Lastly, they measure productivity to get greater visibility into the bottlenecks and inefficiencies in their development processes. That enables them to isolate and fix problems more quickly than competitors. As the saying goes, “you can’t fix what you don’t measure.”

Organizations failing to rigorously benchmark their R&D performance often fool themselves into thinking their productivity is improving. It might be improving, but the question is whether it’s outpacing the competition.

Semiconductor organizations ignoring productivity measurement and benchmarking do so at their peril. Most will eventually cease to exist because they consistently miss schedules and have fewer new products coming to market. Know any?

Originally published at: http://www.eetimes.com/electronics-blogs/other/4371065/Why-they-benchmark-productivity#

NimrodO0l1

4/16/2012 6:44 PM EDT

“Larger project teams almost always exhibit higher development throughput, or output per unit of time.”
I suggest that you reference Brooke’s Law:
“Adding people to a late software project makes it later”
There are other phrases in this article that suggest the author believes engineers are a completely fungible resource. As in: If it takes one woman nine months to produce a child then nine women can do it in a month.
Yes, there are certain projects where a lot of manpower is needed. There are a whole lot of projects where a small team of good people can blow away a large team because the large team spends more and more of its time in communications and other overhead.

To follow on the quote I started with: Large teams almost never produce more function per dollar invested.

daleste

4/17/2012 12:30 AM EDT

Nimrod, you beat me to it. One of my best friends always used this when asked if additional resources would improve the schedule. Alas, cancer took him away. I miss him.

RCollett

4/26/2012 7:33 PM EDT

Nimrod,

1. It is irrefutable that larger project teams have higher throughput than smaller teams — but there is certainly a law of diminishing returns. I have observed this first hand on hundreds of projects.

2. Adding people in the middle of a project, as opposed to at the outset, may or may not increase throughput. It really depends on the particular project and situation. In most cases it will, but the impact could easily be minimal. Again, depends on the project/situation.

3. Engineering resources are not fungible. Not sure why you think the article implies they are.

4. Across the industry companies are increasing team sizes because the rise in design complexity is outstripping productivity improvement. Again, irrefutable fact — based on data.

Daniel Payne

4/16/2012 7:56 PM EDT

I second Nimrod’s assessment, and the classic book, “The Mythical Man Month” explains why. http://www.amazon.com/The-Mythical-Man-Month-Engineering-Anniversary/dp/0201835959/ref=sr_1_1?s=books&ie=UTF8&qid=1334620506&sr=1-1

RCollett

4/26/2012 7:37 PM EDT

Mythical man month is alive and well — to a point. Adding people does not increase througput linearly, but it does increase it. The issue is determining the point of diminishing returns. But most R&D organizations don’t get bogged down in such questions, because lack of resources does not give them the luxury to think about the possibility of adding too many engineers to a project.

Neo1

4/17/2012 2:27 AM EDT

I agree with the author and ofcourse it comes with a caveat that putting more won’t bring out more after a point. Knowing which is that point is where the excellence of management lies. They, like everybody else get reactive when pressure builds up and start ignoring the caveat.

KB3001

4/17/2012 3:48 PM EDT

The law of diminishing returns!

RCollett

4/26/2012 7:39 PM EDT

Please elaborate.

RCollett

4/26/2012 7:38 PM EDT

Precisely. Thanks for your comment. RC

prabhakar_deosthali

4/18/2012 1:32 AM EDT

In my opinion the productivity of a group is governed by the law of logarithmic returns. which means to if you increase manpower ten times then your productivity will be only doubled!

RCollett

4/26/2012 7:43 PM EDT

Wrong. Productivity will consistely decline as team size increases. I think what you meant to say is that increasing manpower by 10X will not result in a 10X increase in throughput. That’s absolutely correct. The increase depends on the quality of the management of the team (and organization and company). The name of the game is to minimize the decline in productivity as team size increases.

t.alex

4/18/2012 7:39 AM EDT

Sometimes the productivity comes from more complex factors such as skillset, cohesion of cross functional teams, etc. There are no hard rules for this, and of course benchmarking or measuring is very subjective right?

RCollett

4/26/2012 7:46 PM EDT

Productivity indeed is a function of skillset, corss functional teams, etc. (e.g. tools, methodolog and management).

Not sure what you mean by hard rules.

Benchmarking is NOT subjective — my firm has been doing it for over 15 years on over 3,000 IC and embedded SW projects.

Frank Eory

4/19/2012 6:25 PM EDT

Be careful what you measure and how the measurement incentivizes or disincentivizes your engineers.

I remember many years ago a project in which management decided lots of productivity metrics should be collected. For IC designers, the metric was transistors per hour, much as for software engineers it was lines of code per hour. Heaven help you if you were doing RFIC design, where your entire design is a relatively small number of transistors!

But for us digital guys, especially those of us with a lot of DSP/datapath content, the metric was hilarious.

Watch me code up tens of thousands of transistors in seconds:

wire [31:0] x, y;
wire [63:0] z;

always @ (x or y)
z = x * y;

Voila! A 32×32 bit multiplier in about 15 seconds. Ok, I haven’t synthesized it yet, but synthesis will complete in less time than it takes me to go get a cup of coffee.

Since DSP designs require many multiply and add operations, I can blow up the die size in no time by dropping down hardware multipliers every time an algorithm requires a multiplication.

Yeah, but we all know that’s not the optimum way to architect an algorithm-on-silicon. But hey, look at my transistors per hour metric!

In the words of one of my favorite Dilbert cartoons from years ago, in which financial incentives were tied to productivity metrics: “I’m going to code me a new minivan!”

RCollett

4/26/2012 7:52 PM EDT

Hi Frank,

Using transistors to measure design complexity is wholly inaccurate — I couldn’t agree more. I wrote an article on how my firm does it, which is a production-proven approach applied to several thousand IC projects. Here’s the URL

http://www.eetimes.com/electronics-blogs/other/4210492/How-complex-is-your-chip-design-

The Best Laid Plans of Mice and Men

Ron Collett — Wed, 18 Apr 2012 10:44:36 +0000

Last month on these pages I discussed “The elephant in the corner“–the wholly unrealistic IC development schedule nobody dares openly question. In truth, the situation is often much worse than I described. Usually it isn’t just one elephant in the corner, there’s a herd—a portfolio of projects. In fact, one of the most insidious problems of portfolio management is the failure to adequately verify that project plans are realistic. Because most R&D organizations lack a reliable verification capability, most portfolios end up in chaos—indeed, “the best laid (portfolio) plans of mice and men often go awry.”

With that in mind, how many chip design projects is your R&D organization currently working on? If it’s a handful or more, my guess is that many will miss schedule—too many projects, too few engineers. When projects are understaffed, the implicit assumption is R&D teams will compensate by achieving enormously high productivity. It doesn’t happen. Instead, much of the portfolio slips schedule—often by a lot.

Oversubscription by more than 40 percent is common in the semiconductor industry. That means R&D organizations needs at least 40 percent more engineers to meet the portfolio’s time-to-market objectives.

Perhaps it’s an open and shut case of wanting to “have one’s cake and eat it, too.” There simply aren’t enough engineering resources to handle all the business opportunities marketing and senior management pursue. But that inconvenient fact rarely prevents major over-commitments. Instead, most projects simply end up underfunded. “Book the business now, and we’ll worry about on-time delivery later,” seems to be the common refrain in the industry.

R&D organizations can offset portfolio under-resourcing in a number of ways:
(a) reduce the scope of projects—difficult to do because markets and customers demand maximum functionality & performance, and likewise, competitors are constantly upping the features/performance ante; (b) re-sequence the portfolio’s execution pipeline by delaying certain projects’ start dates (“resource leveling”)—it’s done sometimes but typically not enough to have much impact; (c) “encourage” the R&D organization to work longer hours, which increases engineering throughput—it happens quite a bit, but engineers burn out and productivity often declines; (d) the R&D organization can attempt to boost its productivity; unfortunately, productivity often must increase far more than humanly possible within the target time horizon, because chip design complexity is increasing faster than productivity typically improves.

So what’s the solution? First, organizations must be far more discriminating in selecting business opportunities and projects. That puts the onus on marketing organizations, as well as senior management, to gain greater insight into the markets they serve. To this end, marketing should establish a robust process for quantifying the likelihood of achieving its forecasted revenues—assuming R&D meets its commit dates—and it should present those (realistic) probabilities to senior management. Second, productivity implied in project plans must be measured at the outset of each project, and projects assuming unrealistic performance should be flagged. Implementing this strategy leads to streamlined portfolios whose projects meet schedule and revenue targets. Moreover, both the engineering and marketing organizations become more accountable for meeting schedules and forecasts.

Originally published at: http://www.eetimes.com/electronics-blogs/r-d-roi/4237939/The-best-laid-plans-of-men-and-mice#

Comments

WW Thinker

3/13/2012 2:03 AM EDT

Too many companies today are still engineering-driven despite of their emphasis on the need to understand the market well.

no.name_#4

3/14/2012 5:45 PM EDT

Hum, “Too many companies today are still engineering-driven despite of their emphasis”; I can agree that leadership is a vital ingredient in a companies success.

I could use some of that yesterday style leadership!

I have seen and worked for companies that are “Engineering Lead” (think Intel, Google) and those that are now “MBA Lead” (think Axcelis, TI, HP).

It seems inevitable, when a company helm is handed to a MBA, the business was inevitably parceled out and shriveled back from it glorious past.

In an industry that grows by staying on the leading edge only with technical vision, a vision for products that win market share and revenue growth; it’s my impression, MBA’s universally lack technical vision and direction. They are good followers, with lots of business discipline; but crappy leaders that can’t lead with a physical product innovation!

Not to say that Eng Lead companies are always successful, they aren’t. They are just more often the leading edge, remarkable success stories that people talk about, then an MBA lead company ever was.

Mark.Rackin

3/19/2012 3:35 PM EDT

One word: MOTOROLA. When led by the founder, Paul Galvin, later (after Paul’s tragic death in the mid 1950s) his son Bob Galvin, the chairs were primarily businessmen. However, they “knew what they didn’t know” and thus relied on a very strong core of engineers to actually run the company. Dan Noble was what would today be the CTO, and the presidency was held by a series of engineers who turned out to be superb leaders too. The sorry decline and fall of this formerly great company started when the 3rd generation (Chris Galvin, who as an MBA was CERTAIN that he knew everything) took over.

Frank Eory

3/13/2012 2:05 PM EDT

You hit the nail on the head with “organizations must be far more discriminating in selecting business opportunities and projects.”

The simple truth is there are far more chips that COULD be designed than the number that WILL be designed, and verified and manufactured, qualified, etc. Deciding what to invest in and what not to invest in is the most important phase of a new product development. And sometimes projects need to be killed to free up resources for a different project that has a better chance of making money.

Robotics Developer

3/13/2012 5:21 PM EDT

I can say from almost 2 decades of experience another schedule killer is scope creep! More often than not, the schedule was too aggressive and the team under resourced (people/machines) but most importantly, marketing/sales insisted on adding a new feature or “just tweaking” the specifications with no regard for the impact to the development. I have seen changes ripple into the pipeline depth, chip size, layout, power, memory needs, pinout and the list goes on. The end effect on far too many designs was the short circuiting of the testing/verification step and an attempt to “speed things up” schedule wise by committing to a layout before all the verification runs were done (or even started). Normally, there were bugs found and redesign/layout needed which set the project back further than if they had finished verification. Upper management is a primary cause of this error and should be held responsible. Most often, the designers were left holding the bag!

DickH

3/13/2012 5:41 PM EDT

to Ron Collett
the best-laid plans of mice and men gan aft agley in the original Scots. ‘oft go awry’ is a decent translation, fine, but keep the mouse / man word order, will you?

Neo1

3/13/2012 11:06 PM EDT

This is akin to “Who will bell the cat”

RCollett

3/14/2012 1:09 PM EDT

Wikipedia explanation of “Who will bell the cat”:

“The Fable concerns a group of mice who debate plans to nullify the threat of a marauding cat. One of them proposes placing a bell around its neck, so that they are warned of its approach. The plan is applauded by the others, until one mouse asks who will volunteer to place the bell on the cat. All of them make excuses. The story is used to teach the wisdom of evaluating a plan not only on how desirable the outcome would be, but also on how it can be executed. It provides a moral lesson about the fundamental difference between ideas and their feasibility, and how this affects the value of a given plan.”

Neo1, thanks for the comment — great comparison!

Grant Martin

3/15/2012 6:20 PM EDT

But you need to finish the Scottish connection between “the best laid plans o’mice and men gang aft agley” and “Bell the Cat”.

A wee bit more in Wikipedia finds : “Archibald Douglas, 5th Earl of Angus (1449 – October 1513), was a late medieval Scottish magnate. He became known as “Bell the Cat”. ” and later “In 1481, Angus became Warden of the east march, but the next year he joined the league against James III and his favourite Robert Cochrane at Lauder. Here he earned his nickname by offering to “bell the cat” – specifically, to deal with Cochrane – beginning the attack upon him by pulling his gold chain off his neck, and then ordering the hanging of Cochrane and others of the king’s favourites. (The phrase “to bell the cat” comes from one of Aesop’s fables, “The Mice in Council”, and refers to a dangerous task undertaken for the benefit of all.)”

Now the circle of cross references is complete!

This is especially relevant for all references to Scotland at the point in time where the Encyclopedia Brittanica has ceased publishing its printed edition. To quote the Usurper again:
“The Britannica is the oldest English-language encyclopædia still being produced. It was first published between 1768 and 1771 in Edinburgh, UK”
or for those in the know, Edinburgh SCOTLAND.

Is it really true that all stories end up with a Scottish connection?

Grant Martin

The Elephant in the Corner

Ron Collett — Tue, 31 Jan 2012 07:42:46 +0000

Why do so many IC design teams commit to development schedules they know are not possible to meet? I ask this question because it’s such a common occurrence in the semiconductor industry. (Don’t read this article if you never miss schedules.)

Schedule misses are so common as to be an epidemic. It’s as if unrealistic project plans are part of the DNA of the chip industry.

Design teams are loath to complain too much about pie-in-the-sky plans. That’s because they gain little by raising red flags, even though they end up shouldering much of the blame when projects miss schedule. Moreover, complaints are often met with resistance by some of the organization’s stakeholders. It’s just better to play along with the charade, as it increases the likelihood their project plans will get funded.

Once published, such fanciful schedules and resource plans become officially sanctioned propaganda. Just about everybody knows their nonsense, but nobody dares to talk about those big elephants sitting in the corner. At least not until it becomes apparent that the tapeout date will slip—often by months. When it becomes clear that a particular project will badly miss schedule, the organization can collectively and plausibly deny it had any clue that the schedule was unrealistic.

So who’s part of this conspiracy? The genesis is usually in the engineering organization but quickly works its way to marketing and senior management. It starts in engineering because project managers know that submitting resource plans requesting significantly more engineers than management will approve can be career-limiting. Mid-level managers don’t get promoted for saying they can “do more with more.” Yet, in order to finish projects within the time defined by marketing and customers, project managers know full well that additional resources are critical. I’ve personally seen myriad SoC projects staffed with only half the engineers they actually need to finish on time.

Does the conspiracy really start with engineering? I think not. More likely it starts with the leadership of the organization—albeit perhaps tacitly. Of course nobody could ever admit to fostering a culture of self-deception, even if unintentional. Likewise, there will never be acknowledgment, tacit or otherwise, of business strategies whose unintended consequence starves projects of resources—even though it’s obvious projects demand more engineering resources to cope with skyrocketing complexity and ever-tightening market windows. I can’t blame management for trying to keep the lid on spending—it’s business. But failure to make the hard decisions about aligning the product portfolio to match resource capacity is fair game for criticism.

Of course somewhere in this mess sits the unfortunate customer. He’s not savvy to the conspiracy—he never sees the elephant in the corner. He gets a glimpse only when it shows up sitting on his conference room table in the form of the chip vendor’s mea culpa. Of course during this meeting, the vendor parades out the usual specious suspects that caused the delay, but everyone knows what really happened: A gross mismatch between resources, design complexity and schedule constraint. The consequence of the mismatch was an assumption of development productivity that far exceeded what the design team could realistically achieve. Semiconductor companies should get their R&D houses in order, as customers are increasingly on the hunt for elephants.

Originally published: http://www.eetimes.com/electronics-blogs/pop-blog/4235164/The-elephant-in-the-corner

RichMo

1/24/2012 1:05 PM EST

I worked for a major semiconductor company for over 14 years. It seems like at least once a year I would get an urgent summons to a scheduling meeting. In order to meet the customer requirement, they had worked the schedule backwards and determined that my tapeout group would have to work over the Christmas holidays. These meetings would usually be held in June or July so I quickly learned just to say “No problem! If you get your design done on time, I will have a team working.” I never had to fulfill my commitment, because the “elephant” was alive and well!

RCollett

1/25/2012 1:49 PM EST

And who says Christmas doesn’t come in July! Thanks for the comment. Ron

zeeglen

1/24/2012 3:54 PM EST

Right you are, Ron. I remember a manager who told me outright when I said we would need more time that he had to fit the project within the time and monetary budget or it would never get approved. Schedule slip was the usual result, mostly due to unrealistically short development time dreams.

Some upper managers could not comprehend that if it takes a cow nine months to produce a calf, one cannot take nine cows and expect the calf in one month. Yes, the math works, but practical development matters rule.

RCollett

1/25/2012 1:50 PM EST

Having facts data, especially industry data, is the best way to convince skeptics. Thanks for the comment. Ron

tb1

1/24/2012 5:59 PM EST

I actually once worked for a manager who had very accurate schedules, and we actually met every schedule for board and system delivery. I still have a hard time believing it, since I had never seen this behavior before or since.

Unfortunately, the software team was managed more conventionally, and the hardware team was always driving them crazy by delivering hardware on time when they weren’t ready for it.

RCollett

1/25/2012 1:51 PM EST

Maybe that happens more in the systems domain than in semiconductor? Ron

jackOfManyTrades

1/25/2012 3:34 AM EST

When I was an IC designer, I soon learned that the best time to book a holiday (vacation) was around the tapeout date. That way, there was zero chance of the holiday coinciding with the actual tapeout.

RCollett

1/25/2012 1:52 PM EST

Too funny! Ron

any1

1/25/2012 9:30 AM EST

In my experience this deception often starts in marketing. They complain that because engineering is not doing its job fast enough the company is not competitive with its peers and so can’t get enough business to make its sales goals. So then management tells marketing that they can promise better lead times to get those orders and then tells engineering that they have to meet whatever schedule that marketing needs to get the business whether they have the resources to do so or not.

R0ckstar

1/25/2012 6:39 PM EST

That’s basically it. Schedules are generally DICTATED, not requested. A true schedule would be an exercise in impartial forecasting, guessing when some other design team of which you have no stake – will finish. but for many reasons, most of which have already been mentioned, scheduling a project that you have a personal stake in is inherently problematic. If you really want to know how long a project will take, ask the other department, or start a betting pool then watch what the odds tell you. I guarantee it will be dead on every time!

AMSURF

1/25/2012 5:22 PM EST

My favorite excuse for the unrealistic schedule is from the Sales or Marketing person who says, “if we don’t deliver it by this date we will lose the socket and be out of the product’s lifecycle!” My response, “let the race to failure begin!”

zeeglen

1/25/2012 6:45 PM EST

Yup!

Marketing: “Company X is making a fortune with this. We gotta have something similar in six months or we will miss the Market Window.”

Engineering response: “Company X has had a whole year to develop this. We need the same. Why the heck didn’t you guys ask for this six months ago?”

skal_jp

1/25/2012 7:35 PM EST

I remember an argument with my manager:
- We need to tapeout in 1 week
- Sorry, with the machine power and license number we have, I need 2 weeks for running all the backannotation simulation
- Can’t do it in 1 week?
- Nope.
- Then only run part of the sim!

Sim results? There was one pattern with a glitch. Of course in respect this Muphry’s law the glitch was on a pattern I ran after tapeout.

zeeglen

1/25/2012 9:02 PM EST

So who got blamed for not running the whole test?

never mind – I can guess

t.alex

1/26/2012 10:21 AM EST

The customer might be the source of all these in some cases.

RCollett

1/26/2012 3:14 PM EST

No doubt that customers frequently request/demand an accelerated schedule, which is natural — they’re reacting to the changing competitive forces they themselves are facing. Such requests/demands trigger a chain reaction within the chip supplier’s organization — marketing, engineering, program mgmt., senior mgmt., etc. must respond — and the common response is naturally “yes, we can do it.” That’s not surprising either. Everybody wants/needs to please their customers. However, what often happens is that difficult decisions are avoided. Instead, the organization pretends that somehow ten pounds will fit into a five-pound bag. RC

Frank Eory

1/26/2012 5:06 PM EST

In my personal experience, this situation has actually improved a lot over the years. Yes, marketing still dictates a market window that must be hit, and the scheduling works backwards from there — from end of qual, to system validation, to first prototypes, fab cycle time, tapeout, verification time, design time, and to product definition and spec writing before design even gets started.

When that back-to-front scheduling process reveals that design has a negative or unrealistically small positive number of weeks or months to do their job, then the number of choices are very few. If a re-evaluation of the lifetime revenue based on a later market entry still makes business sense, proceed with a new schedule that has the later date. If the resourcing was intentionally lean, add headcount to appropriately resource the project and/or re-define the feature set & scope if a lesser product can still satisfy a segment of the market by meeting the market window.

If none of those business tests pass, then the right answer is to kill the project and put your engineering resources on a different project where you have better odds of making money. Sales will complain that “we’re going to lose that socket,” but the reality is that you have already lost that socket before you even started the project…because you waited too long to start.

RCollett

1/26/2012 5:29 PM EST

Frank, what you describe of course is a rationale approach to a challenging situation. However, my observation — based on the Numetrics customers, which is fairly substantial — is that the situation is actually getting worse. It’s because the enormous competition in the semiconductor industry. We need look only at the industry’s M&A activity during the past few years, as well as the companies forced out of business (e.g. the latest being Trident, which recently filed for bankruptcy). The greater the competition, the greater the need for best-in-class management.

prabhakar_deosthali

1/27/2012 6:01 AM EST

The main reason why schedules are dictated and not requested is that the management thinks that the engineering is always adding a lot of cushion to the timelines to be able to work in a relaxed manner.Mnay of the engineering guys actually do it so that after the squeezing out by the management they get the REAL schedule .

RCollett

1/27/2012 12:57 PM EST

Prabhakar,

So you’re saying that in your experience the engineering organization frequently pads the schedule (timeline)?

Charlie_Edmondson

1/27/2012 3:30 PM EST

There is another side to this problem. A few years ago I was on a project (systems, in this case) where I was the chief hardware designer. My manager was an expert on schedules and budgets, a wiz at anticipating senior managements requests, and an all around good guy. Our hardware team was always on time, on budget, and had a lot of fun.

The SOFTWARE side, unfortunately, was run by a PHB who was always promising things he couldn’t deliver, was way over budget, always behind on the schedule, and continually complaining he didn’t have enough resources, money, people or time.

When the phase of the project I was working on came to a close, the company laid off the entire hardware design team, and spent the next year ‘persuading’ the hardware manager to quit. Why? Because he had OBVIOUSLY been featherbedding his budgets and schedules, and not driving his people hard enough. They promoted the software manager… again!

peinal

1/27/2012 3:34 PM EST

If you think this only occurs in semiconductor design, you’re sadly mistaken. I’ve worked designing HW and SW in military/aero, telecomm, and govt and they all have the same elephant in the room. I once refused to sign a proposal that the govt. required for the preparers. I refused because my initial conservative estimate of 10K hrs was cut to 4K hours on a project that would’ve been 3x more complex than the last one we did (which took 6K hrs). See what I mean?

zeeglen

2/1/2012 6:36 PM EST

A way around this is to multiply your conservative estimate by four times, then when the bean counters divide by four you get the time you really need.

Unfortunately the bean counters eventually catch on to this, so you must multiply your next estimate by 8, next by 16…

WKetel

1/27/2012 8:15 PM EST

My observation has been that sales or marketing will promise anything to get the order, and then engineering and production are blamed when delivery does not happen as promised. This was in the industrial equipment realm, not chip design. Once, a salesman with integrity told us that he would really like the order, but he could not make that delivery. He told us what he could make. Another company claimed no problem with the delivery date, and then wound up being several days later than the supplier that we did not go with. Worse yet, the transducers cost more and were not quite as good. Their catalog got a big red “DO NOT USE” note at that point. I seldom forget a betrayal, and I do take it personally.

End of the Free Ride

Ron Collett — Tue, 25 Oct 2011 00:31:10 +0000

According to Pagemill Partners, a well-known Silicon Valley venture capital (VC) firm, the number of semiconductor companies spawned with VC funding has been steadily declining for nearly a decade. In 2003, VCs gave life to 63 new chip companies. Last year the number was 13. It’s a trend that promises to reshape the semiconductor industry. (Note: the figures reflect companies formed in North America, Europe and Israel.)

Established chip companies planning to expand via acquisitions should take notice. Fewer start-ups mean two things will likely happen. First, a feeding frenzy will ensue for the handful of hot start-ups demonstrating impending dominance in their respective markets. Second, limited supply of these successful upstarts will force would-be suitors to rely more heavily on their own R&D organizations for new product development and revenue growth.

The problem with relying on internal R&D is that many executives will learn the hard way that their development organizations are not up to the task. They’ll discover R&D initiatives plagued by low R&D productivity, poor schedule predictability and missed market windows. Division by division, these groups probably will be shuttered or sold off to bargain hunters.

VCs are turning away from semiconductor investments because the return-on-investment (ROI) simply isn’t what it once was. Buyers won’t pay the risk-adjusted price premiums VCs demand. Today, some VCs claim their investments are nothing more than R&D subsidies for established semiconductor companies—and the free ride is over.

Imbalance in the ROI equation stems from the increasing investment necessary to develop complex chips—upwards of $100 million for a system-on-chip IC. For investors to part with that amount of cash, they must see a clear path to a $500 million to $700 million liquidity event. Buyers are loath to pay those prices in all but a few cases. That’s because they don’t think they can extract the requisite sales revenue from the end market to justify the purchase price—often due to cut-throat competition, which is pushing down revenue and margins faster than ever before.

On the other hand, buyers do seem willing to pay $150 million to $250 million for winning start-ups, which means that if VCs serve up companies whose cumulative investment is $30 million to $40 million, we’ll see a steady stream of win-win deals. Toward this end, VC’s are funding specialty chip companies that require substantially less capital, such as those developing analog, RF and mixed-signal chips.

VC firms raising exceptionally large investment funds will probably sideline themselves from the semiconductor game—$30 million to $40 million investments are usually too small for them to justify, especially since the investment is often divided among several firms. But that simply means smaller VC funds will take up the slack.

Another path investors will surely pursue is off-shoring R&D to lower cost geographies, where the size and quality of the labor force has been growing during the past decade. This will reduce start-up costs by 25 percent to 50 percent, or even more. But it will be quite a while before the supply and demand for these offshore-based start-ups comes into balance. Between now and then, semiconductor companies will need to rely heavily on their own R&D organizations.

Originally published at: http://www.eetimes.com/electronics-blogs/r-d-roi/4229889/End-of-the-free-ride#

The Realities of IP Reuse

Ron Collett — Wed, 24 Aug 2011 21:22:55 +0000

Long touted as a silver bullet, IP reuse often fails to live up to expectations when it comes to increasing semiconductor R&D productivity and throughput . That’s because most IC development teams fail to recognize a critical non-linear relationship exists between the amount of circuitry they modify or “improve” in pre-existing IP blocks and the effort the engineering team expends in making those modified blocks operate properly in the target IC. Bottom line: small changes can have a disproportionate impact on project effort. Not being fully cognizant of the specifics of this non-linear behavior is a common trap into which myriad engineering teams unwittingly fall.

Conventional thinking, for example, might presume that modifying a mere 20 percent to 30 percent of a block translates into a relatively modest amount of engineering effort, compared to the effort the team would expend if creating that block from scratch. After all, most of the block remains untouched, has already been verified and perhaps even implemented in silicon. But therein lies the rub. Because of the non-linear relationship, the 20 to 30 percent change (or even less) often translates to engineering effort far in excess of what the project team anticipates.

Frequently, the additional effort is significant. For instance, when teams modify upwards of 40 percent or 50 percent of a block, the total effort expended to make the block work in the target IC can be equivalent to designing it from scratch.

Yet the biggest problem is not the additional effort, but rather it’s that it’s unexpected, which is a common root cause of schedule slip and poor predictability , as well as low R&D productivity. It’s an insidious problem that often disrupts the entire project development pipeline, as resources fail to roll in a timely manner from one project to the next.

Each family of circuit blocks has a unique set of non-linear reuse curves—analog, RF, logic, processor, memory, etc.—and the curves vary with respect to whether the particular IP is hard or soft and the amount of its verification suite reused. When teams apply the curves during the project’s planning phase, the impact can be tremendous: dramatically reduced schedule slip, greatly improved predictability and noticeable productivity gains, all of which go right to the financial bottom line.

Originally published at: http://www.eetimes.com/electronics-blogs/r-d-roi/4218842/The-realities-of-IP-reuse

Comments

AlPothoof

8/17/2011 6:26 PM EDT

The numbers I have seen in the software industry indicate that if you need to modify as little as 25% of the code, you are ahead to start from scratch; the modifications will take longer and cost more.

John.Gyurek_#2

8/18/2011 9:35 AM EDT

All true but the real point here is that if you don’t have a well defined need with standard or well defined I/O that does NOT require modification you are shopping in the wrong aisle. Design reuse is different than integrating an IP block. An experianced project manager should know the difference and schedule accordingly.

Duane Benson

8/18/2011 11:58 AM EDT

I don’t think the software industry gets enough credit in this regard. Consider that language and OS libraries are really just code reuse. If each developer had to hand code all of the functionality that lives in libraries, then we could say that IP reuse doesn’t really exist.

As it is, we’re typically only looking at the in-house coding and ignoring all of the externally developed code.

ndancer

8/18/2011 3:28 PM EDT

I stole this from somewhere – don’t remember where … sometimes, the best IP reuse is to recover the disk space …

Does EDA Matter Anymore?

Ron Collett — Wed, 29 Jun 2011 19:56:52 +0000

Of course electronic design automation (EDA) matters! It’s indispensible to chip design. However, the more important question is whether EDA is keeping pace with increasing IC design complexity. In most cases, the answer is no. Design complexity is increasing much faster than productivity. How do I know? Average development team size continues to grow.

So where does that put EDA? Perhaps the best place to look is the Design Automation Conference held a few weeks ago in San Diego. The venerable 48-year-old show played host to nearly 200 vendors, staged myriad panel sessions and technical papers, and attracted thousands of attendees. But as far as I could tell there were no earth-shattering tool breakthroughs portending to reverse the tide. Maybe you saw something I missed—let me know. Naturally, vendors announced plenty of new products, many of which will undoubtedly boost productivity. But none are likely to obviate the need for ever-larger teams, which is the current prescription for declining relative-productivity.

Engineers battling on design’s front lines know all too well that EDA advances aren’t keeping pace. But in many chip companies and businesses senior executives remain blind to it. A great many seem to be living in the past, neither seeing nor wanting to see that the tired strategy of just pushing the R&D organization a little harder won’t work.

Perhaps the situation is akin to the anecdote about the frog resting comfortably in a pot of cold water whose temperature gradually rises to a boil. The temperature climbs so slowly that the submerged amphibian never perceives the danger. The poor critter fails to jump out and is cooked to death. The story serves as a metaphor for what’s happening in the C-suites of many semiconductor companies and chip businesses. Their executives simply don’t perceive the danger. I’m not sure why, but perhaps they don’t want to hear that the water is getting very hot, so their trusted lieutenants don’t inform them? Maybe they hear but don’t want to acknowledge it? Maybe it’s a combination of the two? For whatever reason, many aren’t taking the necessary action.

I wouldn’t expect C-suite executives of large semiconductor organizations to see first-hand how challenging new product development has become. How can they? They’re busy running the company. Instead, they must rely on those under them. Only engineers and managers battling in the trenches truly see what’s happening. Enlightened executives foster cultures in which lieutenants are encouraged to speak up about hot water (and the limits of EDA)—and they’re heard. Other cultures, well, you know the story of the frog. Which one is yours?

Originally published at: http://www.eetimes.com/electronics-blogs/r-d-roi/4217278/Does-EDA-matter-anymore-#

Comments

KarlS

6/28/2011 5:45 PM EDT

The tools are based on those used some 25 years ago and the approach has been to just get a bigger hammer and hammer the HDL harder. HDLs are a description language based on the fact that logic can be described by a flow chart. So flow charting was used because programmers could follow the flows and do basic synthesis. Static timing analysis came along at about the same time so that functional(cycle accurate) simulation was used. But the process was modified to use HDL and timing simulation. I think that that was oversold so that it continues today. We just had a long discussion about which language really would be good for design. So far LabVIEW with hierarchical schematics which is based on a data flow programming language has a lot of traction. EDA that does not help the designer tie things together is not a tool — and we have too many of them floating around. The discussion is in the LinkedIn FPGA Group.

Death of the SoC

Ron Collett — Thu, 12 May 2011 21:20:11 +0000

Rumors of the SoC’s impending death have been popping up in the semiconductor and systems industries. Are they exaggerated? Not entirely. A decreasing number of companies are investing in system-on-chips (SoCs). Likewise, the number of concurrent SoC projects that typical R&D organizations can undertake is shrinking. The reason: soaring design cost and poor schedule predictability .That makes SoC development increasingly difficult to justify. But does this foreshadow the SoC’s complete demise? I doubt it, but these factors will surely chase more players from the market and drive greater use of alternative solutions.

A look inside the venture capital community reveals a bellwether of the trend. During the past five years, a declining number of firms have shown interest in investing in SoC start-ups (although a lot of money has poured into programmable devices aiming to unseat Xilinx and Altera). Only when the SoC’s risks can be significantly mitigated and revenue projections credibly defended will investors even consider the opportunity. Even then, it’s a tough sell.

Up until a few years ago, a customer of mine (chip company) routinely had six or seven concurrent SoC projects underway. Today, the number is a mere three, with a combined development cost of is $150 million to $200 million. To justify the investment, the three products must garner more than a billion dollars in sales revenue. The company has a good chance of meeting its targets, because it consistently hits its development schedules, but there is no guarantee that the expected volumes will fully materialize. In other words, even with best-in-class R&D execution a nd predictability, market uncertainty makes it a risky bet.

Systems companies developing their own chips (ASICs) encountered a similar situation starting in the second half of the 1990s. High cost, complexity and risk made ASIC development prohibitive except among those whose end-products boasted high profit margins and volume. The requirement persists today. Not surprisingly, many systems houses still developing their own ASICs struggle to rationalize the cost, especially those with poor schedule predictability.

Engineering labor consumes much of the SoC’s development cost—team sizes typically range from 100 to 200 engineers. Such large teams result directly from the inability of R&D productivity to keep pace with growing design complexity. Simply put, to offset falling relative-productivity, more engineers are needed to achieve the throughput required to meet time-to-market constraints. More engineers means higher cost–although many companies attempt to reduce costs by off-shoring development.

Cost mitigation tactics notwithstanding, I suspect that we will continue witnessing the decline of the (dedicated) SoC, except among those companies boasting both market vision and consistent R&D excellence, which means delivering high-margin product on-time and within budget to high volume markets.

Originally published at: http://www.eetimes.com/electronics-blogs/other/4215872/Death-of-the-SoC#

Comments

resistion

5/10/2011 1:57 PM EDT

Only large established companies who strive for chip supply chain independence will continue SoC development.

daleste

5/11/2011 11:03 PM EDT

Good article Ronald. I agree with what you wrote. During the 80′s and 90′s we were able to build custom devices for large customers. We needed at least $1M revenue to start a design. As the process continues to shrink, the design and development costs have gone up too much for customers to bear. Unless the system can’t be built any other way, it is not likely that a custom SOC is warranted. Most IDE’s are building catalog SOC’s with broad market appeal. The only way to get enough volume to reach break even is to have multiple customers in multiple markets. That’s for large SOCs. I do think there are still many opportunities for smaller devices especially analog and mixed signal.

nosubject

5/12/2011 10:28 AM EDT

“Today, the number is a mere three, with a combined development cost of is $150 million to $200 million.”

How to get the estimation of $150~200M development cost?
The cost of 100~200 engineers per year is about $20~30M. Beyond that, the major costs are IP cost, license cost, fab and tools. Are those costs more than 80% of the SOC development cost? (150M-30M)/150M = 80%.

RCollett

5/12/2011 5:20 PM EDT

SoC development cycles are typically around two years, so the $20M to $30M labor cost needs to be doubled.

nosubject

5/12/2011 7:40 PM EDT

But SOC developments are pipelined. At the end of the 2nd year, the 1st SoC is validated and into the production; the 4th SoC is in design; the 2nd and 3rd SoC are in verification and bringup.

2years development cycle cannot justify the high labor cost. Instead, because of the design pipeline, the labor cost is lower than the $20~30M I put there.

Raul Lopez

5/16/2011 12:09 PM EDT

Unfortunately the pipeline seldom happens with such efficiency. Design engineers are often tied up in documentation, SW bring-up, early-access customer bugs and other less-than-ideal tasks for at least two quarters after tapeout.

NickLethaby

5/12/2011 9:14 PM EDT

Don’t forget about the software. Typically customers are expecting Linux, Android, and WinCE ports along with some RTOS ports as well. With today’s SoCs having 30 or more peripherals, including graphics, multimedia, and protocol accelerators that need to integrated into hihger level stacks, you often need dozens of SW engineers as well. Furthermore with the semiconductor-supplied software now running into tens or hundreds of megabytes, the application support burden becomes significant as well.

cliffnotes

5/14/2011 4:21 PM EDT

The way to solve the problem is automation. Why use a shovel when you can use a bulldozer? www.analograils.com/why

resistion

5/14/2011 7:40 PM EDT

The problem seems to be customization rather than SoC. SiP wouldn’t solve the problem either.

chipdude

5/16/2011 11:12 AM EDT

Perhaps 3D IC technology will improve the productivity, reduce costs, and still allow for high levels of functionality. It looks very promising.

Raul Lopez

5/16/2011 12:15 PM EDT

I agree with Ron in the decrease of SoC tapeouts. In the video ASIC industry there is a strong consolidation going on that is going to increase as Intel, AMD and NVIDIA add more HW-accelerated support for video codecs and video processing.

rickclucas

5/16/2011 6:54 PM EDT

Another major factor is that SOCs have not really been complete Systems, only large chunks, as time moves on we are seeing higher levels of integration therfore reducing the number of SOCs in an actual system, eventually we will have only one per system with as much of the other componets part of it as well.

In Search of Best-In-Class R&D Organizations

Ron Collett — Fri, 15 Apr 2011 05:39:36 +0000

Competition among semiconductor companies has become super-heated, and R&D excellence has never been more important to establishing competitive advantage. But how do you know if an organization’s performance is best-in-class, especially that of a competitor? Such accolades are often anecdotal and based heavily on perception, a few unsubstantiated metrics or the halo created by the company’s strong financials. High revenue masks much and distorts even more.

Although digging deeply into the numbers seems like the best way to get confirmation, most semiconductor companies do a poor job of tying R&D costs of projects and product lines to corresponding sales figures. Even with good cost-accounting, revenue numbers are easily skewed by the impact of a strong sales force, heavily financed marketing campaigns and long-term customer-vendor relationships. Financially successful companies often have all three working in their favor, which obscures the true efficacy of their R&D organizations.

Even when verifiable metrics such as cycle time are available, it’s still easy to be misled into believing a competitor’s R&D is best-in-class. For example, let’s say a development organization boasts short development cycles and consistently beats competitors to market. The knee-jerk reaction—its R&D must be best-in-class. But peeling back the onion can quickly reveal they staff their projects with a more engineers than the norm.

Putting more resources on a chip project increases development throughput (until the point of diminishing returns), and high throughput is the key to short cycle time. It measures the project team’s output per unit of time (e.g. a week). But there’s often a price: Higher staffing almost always yields lower productivity —less output per individual—unless the team performs at best-in-class. Best-in-class teams avoid this phenomenon.

Development productivity suffers as team size increases because coordination and communication requirements increase. More people beget greater managerial overhead.

Maintaining high productivity in the face of expanding team size reflects both superior management and design capabilities. It’s a core competency whose importance is growing rapidly because IC team sizes are on the rise—to keep pace with soaring design complexity.

Determining whether a particular project is best-in-class thus requires measuring both its throughput and productivity. Best-in-class R&D organizations boast consistently high productivity across the full range of team sizes.

Independent evidence confirms the paradigm’s validity. Projects with above industry-average throughput and productivity wield a commanding edge over the norm: cycle times are 23 percent shorter, they have 10 percent fewest spins, first-time silicon success rate is twice as high, they have 51 percent better schedule performance and 68 percent lower cost per unit of output.

Originally published at: http://www.eetimes.com/electronics-blogs/other/4215108/In-search-of-best-in-class-R-D-organizations

Comments

FastDev

4/26/2011 8:36 PM EDT

Great analysis of the difficulties of measuring R&D productivity. The challenge is getting easier, however, with the advent of new toolsets and methodologies, that fundamentally change the way organizations conduct R&D. We’ve seen customers using a combinatorial R&D approach achieve results in weeks that would have taken months or years with traditional methods. The resulting improvement in efficiency enables project timelines to be met or pulled in without assigning additional staff. While this doesn’t provide a direct mechanism for benchmarking against competitors, it does allow you to measure internal progress, both in terms of cost and ultimate success in bringing new technologies to the market.
John Behnke
VP WW Sales and Marketing Intermolecular

cdhmanning

4/27/2011 5:30 PM EDT

Or, with slightly different terminology, efficiency vs effectiveness,

I don’t know that you can really make useful comparisons. There is far too much variance between different companies and even between different teams in those companies.

For example, how do you compare a team developing an automotive chip (high reliability, high temperatures,…) to one developing a consumer chip (fast, low power). The first team have a lot more emphasis on testing and verification and need to move conservatively but the second group just need to get the next gen chip out there fast.

RCollett

5/9/2011 9:56 PM EDT

Absolutely correct — application of the Best-in-Class paradigm requires comparison of similar kinds of products.

Test_engineer

4/29/2011 9:25 AM EDT

It ain’t Research in Motion (RIM). This outfit is going the way of my former employer, Nortel Networks; namely, down the tubes. Canada should stay away from high tech industries and concentrate on their competitive advantages in producing hockey players and stand-up comedians.

The Politics of Productivity

Ron Collett — Wed, 30 Mar 2011 22:22:25 +0000

Politics and productivity seem to go hand-in-hand in semiconductor R&D organizations. Perhaps it’s natural. No manager or project team wants the low productivity Scarlet Letter. So it’s hardly surprising that ostensibly poor performers use politics to avoid scrutiny.

But are these so-called low productivity projects really poor performers? In fact, many are not. Quite the opposite in fact—they often have high productivity (although insufficient throughput) but are mistakenly pigeonholed because their crime was a missed schedule. Moreover, schedule overrun usually is not due to low productivity.

Instead, it usually stems from an unrealistic project plan—one in which the allocated staffing is not commensurate with the chip’s design complexity, project schedule constraints, and project uncertainties such as spec stability, new mfg. process, EDA tools, etc.

In my experience, unrealistic plans trace their roots to the organization’s executive ranks. Senior management promotes a culture of over-commitment in which R&D takes on significantly more projects than it realistically can handle. The consequence is obvious and ubiquitous: most projects in the portfolio are resource-starved.

It’s an all-too-familiar story: Projects are inadequately resourced, whereupon they miss schedule, senior management thinks the culprit was low productivity, the team runs for cover. Sound familiar? Of course, executive management has no proof to support its (often unspoken but implied) “lousy team” hypothesis, but likewise the team has few facts and little data to defend itself. Who wins that battle? As the story repeats, the organization gets steeped in the art of productivity politics, including subterfuge, avoidance of scrutiny and measurement, and internecine feuds that stems from finger-pointing.

The great irony is that measuring productivity preempts much of its politics. That’s because measurement gives project managers the data they need to prove projects are woefully understaffed and is the root cause of schedule slip, not poor productivity. When managers argue cases using facts and data, they can do so persuasively, often receiving additional resources for their trouble—at least enough to give them a fighting chance to finish within tight schedule targets. They rarely get all the resources they need, forcing them to boost productivity by ten percent or even more. But they’re motivated to go the extra mile—doesn’t management want that?

Executives win when they make wise staffing decisions—they get dramatically improved on-time schedule performance. Likewise, measuring productivity ensures R&D performance improves at a competitive rate. It also yields the facts and data to ensure future projects get staffed (reasonably) properly, and this preempts or at least mitigates the politics of productivity.

Originally published at http://www.eetimes.com/electronics-blogs/other/4214604/The-politics-of-productivity

Optimal Team Sizes for Chip Projects

Ron Collett — Thu, 03 Mar 2011 22:00:26 +0000

What’s the optimal team size for a given IC design project? It’s a question I hear often from engineering managers and senior executives. What they’re actually asking is whether they’re over-staffing projects and therefore wasting resources. Implicitly, they’re also asking “what’s the fewest number of engineers I can put on a given project and still finish on time?” They’re important questions directly impacting R&D ROI.

Projects demand a threshold number of engineers to meet schedule targets. Yet, there’s a point at which adding resources yields little, if any, additional development throughput— the exception is when a project desperately needs a particular kind of expertise or specialist. Although most R&D organizations lack the infrastructure to reliably quantify the number of engineers a project needs (which is why many miss schedule), managers instinctively know there is a point of diminishing return. Additional staffing increases overhead, including communication, coordination and consensus-building. That bleeds development time, lowering average productivity among team members. Each additional resource reduces team productivity—throughput increases, but the issue is by how much?

How should engineering managers determine the point of diminishing returns and therefore the optimal team size for a given chip design project? The answer lies in knowing the relationship between team size and productivity of their organization. Each R&D group has its own ‘productivity vs. team size’ curve (visualize an x-y plot in which the x-axis is team size and the y-axis is development productivity). As team size grows, productivity declines. The steeper the decline, the less of a boost in throughput with each resource added..

Best-in-class semiconductor R&D organizations suffer minimal degradation in productivity as team size grows—their “curves” are almost flat across a wide range of team sizes. That’s one of the reasons they’re best-in-class.

When an R&D organization calibrates itself in terms of productivity versus team size, it gains a huge competitive advantage. Knowing the optimal number of resources to put on projects ensures they are neither over-staffed nor (grossly) understaffed.

These top-tier groups often staff projects based on the assumption that their teams will achieve best-in-class productivity, which means projects are staffed with the absolute minimum number of resources. To hit schedule targets, the development teams’ productivity must be best-in-class.

In sum, adding staffing increases throughput but average productivity declines. The fundamental struggle centers around understanding the tradeoff between lower productivity and higher throughput—how much does productivity fall and throughput rise with each additional resource. The optimal team size for a project depends on the particular R&D organization, the target design’s complexity and the project’s schedule constraint.

Originally published at: http://www.eetimes.com/electronics-blogs/other/4213604/Optimal-team-sizes-for-chip-projects