Improving safety for all road users is at the heart of everything we do at TuSimple. Creating a safe Level 4 Autonomous Driving System (ADS) for Class 8 Semi Trucks requires world-class testing capabilities
To help achieve this mission, TuSimple created the first-of-its-kind Systems Integration Lab (SysIL) that incorporates a fully functional Semi + ADS into a fully instrumented lab. The SysIL adds a layer of safety and test coverage not possible with simulation, test benches, and road tests alone. TuSimple’s SysIL enables a faster path to a safe and feature-complete ADS.
Systems Integration Lab
Left: Truck Under Test Right: Test Lab
The truck operates as an integral part of the lab through an umbilical cable.
Performing safety-critical testing on closed tracks is expensive, requires a large cast of people to travel to a test track, is difcult to perform as repeated regression testing, and incurs all the track testing costs each time.
Additionally, there are things you would never do on a truck intended to return to the road. For example, it would be unsafe to cut into high-reliability power and communication harnesses to simulate failure modes, and then expect that truck to return to road operations. These types of testing must be executed in a controlled, system-level environment. Piecewise testing on isolated Hardware in the Loop (HiL) stands lacks the “integration effect” at the system level. Testing where all the subsystems interact is necessary to nd undesirable interactions you would miss on partial subassemblies.
To meet this challenge, TuSimple took a page from traditional automotive “Bucks”, mock-up vehicles used in a lab for testing, and extended the concept to approximate the aerospace industry’s “Iron Bird” mockup for jets and rockets. By doing so, we created the rst Systems Integration Lab with a fully incorporated ADS capable commercial tractor. We call this Truck in the Loop (TiL), which is the foundation of the Systems Integration Lab. In the SysIL, sophisticated integration work and testing is possible to fully exercise the entire autonomous driving system along with the vehicle platform.
Every finding improves the safety of our ADS. Every new test becomes part of our validation and certification evidence that we are meeting our Safety Case.
–Adrian Thompson, VP of Systems & Safety Engineering
“We hired experts in systems level integration and testing from automotive, aerospace, and defense industries to make this happen. It was a great investment! Every nding improves the safety of our ADS. Every new test becomes part of our validation and certication evidence that we are meeting our Safety Case.” Adrian Thompson, VP of Systems & Safety Engineering.
Why does this matter? Improved safety and quality, reduced schedule and cost.
“A Systems Integration Lab is the only way to test combinations of features, functions, and failure modes, end-to-end, in a safe environment. We artificially create complex situations, nuanced failure modes, and more sophisticated sensing and automation that cannot be replicated on a track or on a partial Hardware in the Loop test stand.” Tim Jones, Director of Systems Commercialization and Next Gen Programs.
A Systems Integration Lab is the only way to test combinations of features, and functions, and failure modes, endto-end, in a safe environment.
–Tim Jones, Director of Systems Commercialization & Next Gen Programs
For example, worst-case stackups are where we test edge cases in combinations to ensure things work correctly when multiple aspects are pushed at the same time. Many of these tests could not be done on public roads, and are difcult to do safely on a closed track.
A Systems Integration Lab is a necessary tool for companies moving beyond research and development and pushing for the true safety critical commercialization of a fully authorized Autonomous Driving System on Class 8 semi trucks.
Applied Learnings from Safety-Critical Industries
The SysIL underscores TuSimple’s commitment and leadership in this space as it continues to evolve its Systems Engineering, Safety Engineering, and Independent Verification & Validation rigor.
Other safety-critical industries already know how valuable a Systems Integration Lab is to the continuous integration of new hardware, software, and firmware. A SysIL enables engineers to quickly identify and resolve gaps in requirements, design, integration, and validation efforts.
Safety Critical Benefits of a Systems Integration Lab
|Always Fully Instrumented||
|Full Complement of Hardware||
|“Test as you Fly”||
|Pause On Fail||
|Integrate Where You Instrument||
A Systems Integration Lab is both Complementary and Supplementary
A SysIL is complementary to other testing modalities because it provides additional condence in features and functions tested in unit testing, subsystem and HiL testing, simulation & regeneration testing, analysis, and track & road testing. It is supplementary because the SysIL enables unique tests not possible on piecemeal testing solutions, with deeper instrumentation, better ability to divide and conquer the verication of subsystems, and better holistic data.
Test Capabilities Added by the Systems Integration Lab
|Test Capability||What Is It?||Benefits|
|Man-in-the-Middle testing||Incorporating external computers and communication cards into the middle of Ethernet, CAN, and other communication buses to defeat, delay, or modify messages.||
|Incorporates relays and control automation techniques to simulate hard-to-handle power failures on an autonomous vehicle. Provides a methodology to test loss of power to any safety critical device. Goes beyond theoretical failure analysis or limited disaggregated HiL test benches, providing empirical evidence of coverage and system wide reactions.||
|Enhanced Data Acquisition||Provides higher delity signal and timing analysis than on-board systems alone.||
|Customized Command and Control of Subsystems||Automated test equipment provides
the ability to connect, disconnect, or reroute signals to specialized measurement, recording, and analysis equipment.
|Accelerate Verification of Failure Monitoring||Automate the injection of every known failure message from OEM faults (sensors, brakes, steering, engine, transmission, etc.), TuSimple generated hardware faults, and software generated monitoring faults||
|Adversarial Testing||Provides a safe place to test “what if” theories. Want to know how your system responds to a loss of communications from the primary to secondary controller? Pull the plug!||
The Systems Integration Lab provides TuSimple with the ability to make modications that would not be safe or possible on a truck intended to return to the road. As such, we are able to get more detailed diagnostic information on signals and timing that we wouldn’t otherwise be able to see.
By automating the collection and analysis of SysIL data, we improve the repeatability of our testing and gain clearly comparable test results from build to build. Using Lean Six Sigma approaches, we can build automated detections of in-spec, but out-of-family test results to identify unintended consequences of changes. By building automated reporting into the SysIL, we can quickly grow our body of evidence that the truck is operating safely.
The SysIL reduces the development lifecycle by unearthing problems earlier, which would be harder to detect or remain dormant and unknown in the system. One of the biggest hindrances to schedule is having to rework features after they have been released. Testing on independent Hardware in the Loop (HiL) test stands takes time and lacks the full system perspective for complete results. Testing on the track takes time to schedule and deploy teams, trucks, and supporting resources, and then time to get the test findings incorporated back into the workflow.
Testing on the Systems Integration Lab can be done whenever required, at the full system level, with automated testing and reporting. Faster, more automated testing with broader coverage and sensing capabilities is the best way to accelerate development and close gaps completely.
While the cost of harvesting a perfectly built-out ADS-enabled truck for a lab is not trivial, the payback period is very short. TuSimple has been actively migrating any track and road testing related to managing component failures to the Systems Integration Lab. We can test and retest for specific failure detection, logging, and responses in a repeatable, reliable, and cost-effective manner
Once we incorporate a test into the SysIL, it’s ours forever. With fewer personnel required to perform testing, zero travel for people and equipment, and faster system-level regression testing and results to developers, the SysIL pays for itself.
A Systems Integration Lab is an absolutely necessary capability for any company building a safety-critical autonomous vehicle.
- Provides system wide connections not otherwise possible
- Enhances integration and test coverage feedback
- Allows testing of failure modes not safe to perform on the track or road
- Pays for itself in both improved schedules and lower costs over time
TuSimple has seen the technical findings and benefits of having a SysIL, proven the efficiency value, and is prospering from the additional understanding that comes from a fully instrumented system. We believe maintaining such a Systems Integration Lab capability should be one of the minimum standards for any company intending to operate autonomously on public roads. We are super proud of our team, and we can see the benefits of their work every day.
Adrian Thompson is the VP of Systems & Safety Engineering, which includes formal Verification and Validation activities. Mr. Thompson brings over 32 years of experience across intelligent systems, chemical engineering, military and commercial aircraft, …
Timothy Jones is the Director of Systems Commercialization and Next Gen Programs. Mr. Jones brings over 36 years of experience building complex testing and enterprise processes for safety critical applications such as military and commercial aircraft avionics, and rocket avionics for the Atlas V and Delta IV, the Space Shuttle, and the Space Launch System (SLS).
The post TuSimple’s Dedicated Systems Integration Lab Enables Safer Path to Level 4 Autonomous Semi Trucks appeared first on TuSimple.