From the simple question “Where is my freight?” an entire telematics and visibility solutions industry was born, dedicated to tracking the physical movement of freight and transportation assets, from intermodal containers to tractors, trailers, pallets and parcels.

Technology startups led by Macropoint, 10-4, project44 and FourKites sold their visibility solutions to shippers, enabling new supply chain optimization initiatives and raising service expectations for transportation providers. Even the transportation management system was fundamentally altered from an on-premise workspace to a cloud-based connective tissue that could easily carry shipment information back and forth from shipper to 3PL to carrier. Beginning roughly five years ago, dozens of integrations between visibility solution providers and TMSes were announced, commoditizing the technology and making basic visibility table stakes for any sophisticated shipper.

Today, the transportation and logistics industry is moving beyond “Where is my freight?” to more complex questions such as “How are my carriers performing relative to each other? Are my own facilities responsible for delayed shipments? What is the probability of this shipment being delayed?”

Finally, a question that is becoming increasingly important to large companies is: “Where are there opportunities to make my supply chain more sustainable?”

The answers to these questions will be derived from the vast data lakes generated by visibility solution providers and 3PLs that have collectively tracked millions and millions of shipments.

A number of issues obscure the question about relative carrier performance. First, how is performance defined? The map of service metrics that shippers care about is a densely overlapping Venn diagram, but it isn’t a perfect circle: Some indicators matter more to certain shippers, whether it’s tender acceptance, on-time pickup, tracking compliance or damage claims. Many shippers create their own scorecards, and 3PLs that want to service their freight efficiently must have the ability to replicate those scorecards internally for their own teams.

But once service is defined, data integrity becomes a problem. Many on-time metrics are self-reported because of the lags associated with manual, face-to-face check-in procedures at shipper facilities. Because check-in lines can form and drivers tell the dock when they “actually” arrive, there’s significant room for error and bending the truth. Tech-forward 3PLs like Convoy use GPS data and geofencing techniques to report on-time delivery and on-time pickup accurately, eliminating the issues that come with self-reported data.

Freight visibility does not only benefit the shipper by giving it insight into carrier performance — it also builds an objective record of the performance of shipper facilities. When Convoy combines OTD, OTP and detention data with its hundreds of thousands of carrier facility ratings, shippers get a triangulated view of their own networks.

In its “Supply Chain Visibility and the Digital Freight Network” white paper, Convoy provided an example of a shipper that is struggling with dock bottlenecks and backups, but because the shipper is working with anecdotes, it can’t identify any specific patterns. In the example, Convoy wrote, “The data uncovers that detention begins to increase at 3AM, builds to a peak between 6 and 7AM, and returns to normal by 11AM. Once time of day is identified, we look at shipment volume and detention cost to uncover that you’re effectively paying an extra $40 per load during the peak hours of 6 to 7AM.”

By rearranging dock appointment schedules to relieve pressure from peak times and exploit slack in off-peak periods, shippers can make carriers happy and reduce their own costs — but only if they can aggregate clean data and visualize it in an intuitive way.

The long-standing goal of visibility solution providers has been to offer predictive estimated times of arrival (ETA): Based on everything we know about this load, the origin and destination, and the carrier, what is the probability of the shipment being on time, or delayed?

Creating a rules-based algorithm to calculate this probability would necessitate the careful weighting of hundreds of variables and would probably never work correctly. But through machine learning, Convoy’s prediction engines take in information about delayed and on-time loads and look for patterns in the data. Machine learning algorithms apply rules to large sets of training data and become smarter over time, simplifying the process of building predictive capabilities.

The sustainability issue — the huge opportunity to reduce Scope 3 carbon emissions by redesigning supply chains — also requires the sophisticated management of visibility data. In this case, operating metrics previously thought important only to carriers have become vital to shippers. Empty-mile percentages, or the percentage of total miles driven that were driven unloaded, is an operating metric used by truckload carriers to measure asset utilization. Because the fixed costs of operating trucks are high and available revenue is constrained by the driver’s hours of service, running empty imposes a significant penalty on carrier performance.

Carriers design their networks so that contracted freight overlaps on the same lane, in opposite directions, or they use brokers to fill their backhauls in order to minimize empty miles. But when a shipper books a truck to pick up its freight and the truck must deadhead to the shipper facility, the shipper is responsible for excess carbon emissions. Convoy helps shippers do the opposite by matching freight to trucks that would already be driving empty on that lane, reducing empty miles and eliminating unnecessary carbon emissions.

By feeding carrier data back to its shippers, Convoy shows its customers how efficient digital freight matching moves more freight with fewer carbon emissions. That helps large companies make progress toward their sustainability goals.

The next generation of visibility solutions is already here, but the technology is not yet ubiquitous or commoditized in the same way as simple tracking capabilities. New solutions around carrier performance, facility network efficiency and design, and sustainability depend on novel technologies that build clean, accurate datasets from the beginning. These solutions require machine learning algorithms to recognize patterns invisible to a human analyst. And they will emerge from digital freight networks like Convoy’s that have the ability to orchestrate a seamless flow of data from shipper to carrier and back again.