Embedded Software Project Benchmarking

Embedded software development organizations look to Numetrics professional services to benchmark their embedded software development capabilities and analyze the development cycle to identify which phases are having the greatest impact on productivity and cycle time performance. As part of the root cause analysis, customers receive a quantitative assessment of their productivity, cycle time performance, throughput, development process quality, schedule performance and reuse strategy effectiveness. Also included is an assessment of whether they’re staffing projects appropriately and how effective they are at managing projects of different sizes–all compared to industry norms.

When benchmarking performance, Numetrics’ consultants leverage the Numetrics Embedded Software Industry Database, which comprises performance metrics compiled from more than 1,200 IC development projects from over 1,000 electronics companies. Only embedded software projects that have been rigorously benchmarked using Numetrics’ proprietary methodology and are certified as such by Numetrics’ professional services are entered into the database. A an overview of the Database is as follows:

Industry Database Profile

  • More than 1,200 embedded software projects spanning 43 vertical segments including the following:
    • Aerospace, Automotive, Avionics, Satellite, Wired & Wireless Communications, Computer, Computer Peripherals, Medical, Display, Electronic Currency, Embedded Control, Portable & Professional Entertainment, Personal Digital Terminals, Photography, Security/Identity, Sensors, Storage, Military, Industrial, Networking, White Goods
  • Project team sizes: range from 1 to more than 600
  • Code volumes: span 1,000 to over 2 million lines of code
  • More than 50 operating systems and 20 programming languages:
    • Operating Systems: Android, ThreadX, Green Hills, FSMLabs, Keil, Linux, Mentor Graphics, Metrowerks, Microsoft, QNX, Quadros, Radisys, Symbian, Texas Instruments, Wind River, Micrium, OSE, CMX, Apple, ST Microelectronics, Unix, and various proprietary OS’s
    • Programming Languages: C, C++, C#, ASP, Assembly, Embedded PASCAL, Java, Real-time Java, LabView, and various domain-specific languages
  • More than 50 main processors from the top manufacturers

Metrics

  • Performance Metrics
    • Project Duration, Schedule Slip, Project Effort, Peak Team Size
    • Design Defect Density, Residual Defect Density, Effort per Defect, Tests per Defect
    • Lines-of-Code/Day, Lines-of-Code/Person-day
    • Customer Requirements, Functional Requirements, Test Cases, Lines of Code
    • Test Case Reuse, LoC Reuse, Requirements Reuse
    • Lines of Code per Requirement; Tests per Requirement
    • Total Project Cost, Cost per New-LoC

Diagnostic Information

  • SW Dev. Environment
    • Number of Design Sites
    • Team Experience
    • Open Source & 3rd Party interaction
    • Spec Changes & Spec Change Rate
    • First Time Challenges
    • Geographic location
  • SW Dev. Methodology
    • Development Model
    • Requirement Tracing method
    • Prototyping strategy
    • IDE
    • Build strategy
    • Verification methodology
    • Release strategy
  • Hardware Platform
    • Platform proprietary content
    • Hardware maturity
    • Processor Type& Processor vendor
    • Number of processors