Anticipate, Advise, Adapt, Act: The Data Deep Dive

Disruptive Analytics – a platform for a better understanding, as described by Dave Bonn

It’s data, data, data all the way: “It’s tomorrow’s oil”; “Big data is the way forward”; “You can’t have enough data”; “Data analytics is the future”.

We see these or very similar expressions in every technical publication, we hear it at every conference these days, no matter what the domain under discussion is. Do we believe it? Does having data really make things that much better? Is it just about collecting more and more data? Do we know why we are collecting it? And perhaps more importantly, are we getting the best out of the data we have?

To look forward we need to consider what has been happening in the past. Centralised systems became commonplace with all sorts of interfaces to data sources, many with bespoke interface protocols and of differing update rates and widely varying data quality. As time has progressed, standardisation for same type devices has improved these aspects significantly but the standards have struggled to keep up with the myriad of new data sources that are being deployed on the transport network. Looking backwards, many central systems suffered from gaps in data collection, erroneous values and a lack of contiguous coverage of measurements. This makes decision-making based on the available data questionable, both in real time and historically when trying to predict future operating scenarios.

“In the past, many central systems suffered from gaps in data collection, erroneous values and a lack of contiguous coverage of measurements. This makes decision-making based on the available data questionable”

The future promised to address these issues by providing a greater density of “easier to collect” data. We hear how the Internet of Things (IoT) will deliver greater benefits in data collection and control capabilities. While this is a different approach, technically is it not similar in the broad sense of collecting data from disparate devices in a central system and making it available to better inform the decision-making process? It is good to see from an operational perspective that today’s smart devices do indeed provide more data enabling operational and maintenance staff to be made aware of transport network operating conditions as well as the device’s status in near real time. Is the deployment of smart devices any cheaper using an IoT approach and does it provide any better data coverage, or is it still down to the capability of the devices and their density on the transport network?.

There is an increase in the use of Edge Computing where data interpretation is taking place closer to the point of measurement. This can only lead to better more reliable actions being taken in a repeatable fashion. The central systems then need only be told of the actions taken. This distributed processing may reduce the amount of data transmitted back to a central location but doesn’t reduce the need for data collection local to that device. Balance this with the increase in crowdsourced data available, social media, connected vehicles etc, where vast quantities of data are available but it still needs to be sorted and filtered to extract the key information relevant to the transport operator at that time, at that location. It is the ability to get quality information that can support operational staff make the best decision that the industry is striving for.

Having sourced the data, cleansed it and packaged it into the format that better support the decision-making process, the central system can then undertake informed analysis on what action is best taken to meet current and future conditions. We see the Intelligent Congestion Management Programme (ICMP) in Sydney seeking to “detect in 5, react in 30”, or in other words detect current conditions and react in a way that prevents adverse network conditions in 30 minutes. This requires a significant change in approach to decision making away from reactive to proactive intervention strategies. Accessing historical data helps, using current data is critical but more so is the need to bring in augmented analytics which enables other impacting factors such as weather, time of day, day of the week, angle of the sun, air quality, presence of surface water, speed of traffic, volume of vehicles, if the road being used as a diversion route, and so on. The list of factors that impact driver behaviour is extensive.

“In the past, many central systems suffered from gaps in data collection, erroneous values and a lack of contiguous coverage of measurements. This makes decision-making based on the available data questionable”

Think of augmented analytics as the closest our machines today can get to the human brain in terms of processing, balancing and reasoning through multiple data sources to confidently predict and plan for the most likely scenario. How does augmented analytics emulate the human brain to surpass less effective single thread data analytics, data cubes and data warehouses? By following the Anticipate – Advise – Adapt – Act sequence, augmented analytics parses which data sources present evidence of value to generate a Prescriptive Option. As mobility services automate, there will not always be a human brain to act on the emerging scenario. The time saved by augmented analytics to quickly prescribe the option to treat a risk or incident can mean the difference between commuter chaos and re-routing for trip optimisation.

“As mobility services automate, there will not always be a human brain to act on the emerging scenario”

To understand how “augmented analytics” can support transport managers, they must first identify the problem they are trying to solve. Too often this isn’t clearly understood at the outset of the problem-solving journey. In common with many other industries, the transport sector agrees that data analytics is a very good thing and that it has the potential to drastically improve decision making (if done properly). Many however recognise that data analytics is not exactly the easiest thing to pull off in the real world. For example, you may detect that traffic using a particular route has increased in the last year by 15 percent but the reason for that change may not be immediately obvious. That is why the use of deep data analytics is key to informing you of the reason for the change so you can react and decide whether you want to discourage the increase or apply the same reasoning to other routes.

Now that the reason that drove the change in driver behaviour is understood, transport managers have an insight that is actionable because it connects directly to an action you can take to address your traffic change challenge. Going forward, your decisions are now based on known factors and an understanding of their impact. This identification of the reason for change required the use of disparate data sets, data cleansing, analysis and the presentation of the results in a real-world action driven format. What augmented analytics does is to relieve a business’s dependence on data scientists by automating insight generation through the use of advanced machine learning and artificial intelligence algorithms done using natural query language.

An augmented analytics engine can automatically go through the available data, clean it, analyse it, and convert these insights into action steps for the transport network operators with little to no supervision from a technical person. It is data fusion without the programming. Augmented analytics can, therefore, make analytics accessible to all. By involving staff with an understanding of the domain but who are not necessarily data scientists, the power of data insight and how it can be best used is given back to those in the business who are tasked with making informed operational improvements: it’s a business pull model.

“An augmented analytics engine can automatically go through the available data, clean it, analyse it, and convert these insights into action steps for the transport network. It is data fusion without the programming”

Further questions remain: from where does this approach get its from? Does it need to go to a data broker who has collected the data already, does it create its own data broker store through data replication or does it need to measure its own data? What is available today is the ability for the augmented analytics engine to interface to data stored anywhere. Be it a legacy store or a real-time source these tools, subject to access constraints on the part of the source system, can access and use any and all available data sources. The concept of creating a “local copy” of the data for analysis purposes is no longer required. The role of “Data Broker”, the lack of which, for the transport domain, has prevented many organisations from accessing the data that would support them to gain a better understanding of what they were trying to manage, is now not an issue. Staff in these organisations can go directly to the source of the data they need without the use of an intermediate facilitator or Data Broker This is very much a disruptive approach in the transport data analytics environment and one that will take some organisations a period of time to get comfortable with but it is the future.

A frequently encountered challenge that will continue to frustrate efforts to make best use of data collected is the constraints many organisations put in place to hinder collaborative sharing of data. There have been may initiatives from governments across the globe striving to make data “open” and this is to be encouraged. Bringing industry along with that same thinking is key. Too often, claims of commercial sensitivity and confidentiality is hampering the wider sharing of data. Yes, we must respect the commercial sensitivity of data sets within competing organisations but perhaps some are over-protective of some of their data and, if it was made “open”, better decision-making by other organisations such as network managers could be achieved.

If augmented analytics can be used to support mobility decision-making, the potential exists to improve government, corporate and private contributions in a wide range of aspects including environment, sustainability, heath, liveability, urban planning and employment opportunities. The use of augmented analytics enables the impact of decisions made to be fully understood taking into account all the variables that impact a decision. The people undertaking that analysis understand the domain and hence better understand the relative impact of the measured environment.

There are organisations that have an understanding of how to enable the augmented analytics approach to release business benefits. The ROI associated with using this new technology will always be an initial question however there is an increasing amount of evidence to show the benefits that can be achieved. Organisations such as Silicon Billabong offer support services on the use of the products available in the market today. Silicon Billabong’s approach is to support organisations identify the “low hanging fruit” in the data world ensuring that early benefits can be both demonstrated and realised for the benefit of the business. This gives senior management the confidence they are looking for to take this approach forward. It also gives those new to the tools the confidence to take on more challenging tasks. Using only the internal data that is currently available for the initial deployment is usually the first step before branching out into data sources in external locations. The rate of deployment of more complex solutions can then be driven by the business using domain-aware staff rather than waiting for data scientists to be hired.

While this disruptive technology still has some convincing to do within some organisations, there is little doubt that augmented analytics will give businesses who embrace the technology as part of their data plans a market edge on their competitors. This won’t be a quick fix to all data challenges but it enables transport operators to better understand the impact different factors have on driver behaviour, enabling the most effective interventions to achieve a desired outcome to be better understood.

Transportation Case Study 1

With the number of variables that impact the outcome of transport movements by both people and vehicles, no two rush hours are the same and as a result no two days of live traffic management are the same. Despite the huge experience of the control room operators gained over many years of handling the unexpected, their reactions to regular events or spontaneous incidents are based on their historical experience, scenario briefings and the support they can get from today’s decision support systems. By combining prescriptive Analysis with Artificial Intelligence in a natural language processing environment, a step change in transport management decision support capability will be delivered. Accurate transport models with real-time data, traffic operators know the current and future network conditions better and can do their jobs more effectively.

Transportation Case Study 2

Whether it is sensor technology or data from vehicles, the IoT deployments on the transport network can only mean more data being available. This data will need to be processed to provide effective decision-making support to operational staff. Similarly Connected Vehicles themselves will require access to large quantities of data whether it comes from on-vehicle sensors or from a central control system to enable them to operate safely. The ability for transport management centres to manage autonomous vehicles in the future will require a significantly greater amount of data which itself will need to be sourced from a number of places meaning augmented analytics will have an even greater role in the management of transport in the future.

Transportation Case Study 3

Most organisations require a Business Case to be prepared in advance of a project being funded. The business case will contain a definition of the cost benefit ratio. This will be calculated to demonstrate the financial viability of a project. If the business case team was able to access data on external projects of a similar nature using augmented analytics it would be possible to produce a more accurate prediction of the cost benefit ratio.

If your organisation serves the Mobility-as-a-Service market, then you know data volumes are growing to a peak never experienced before. The earlier you are in the Mobility value chain, the greater the business benefits augmented analytics can deliver. Accessing crowdsourced data to understand trip patterns quicker than the brain can comprehend is ‘gold’ to planners and financiers who assess project feasibility; joining data to connect transport user behaviour with available modes quickly escalates benefits from ‘social utility’ to a positive economic return. City authorities who use augmented analytics to generate prescriptive options in space planning, maximise amenity for visitors and residents, striking the right balance between rateable value, reduced congestion pressure and cutting nasty externalities that keep people out of the city. Moving decisioning from ‘gut-feel’ to predicated on best available evidence can release over 50 per cent discretionary spend to better directed Mobility-as-a-Service priorities.

Transportation Case Study 4

Most transport managers want to find ways to generate financial returns from projects and operations. Augmented analytics is quicker than the human mind in finding and joining data relationships that impact financial as well as service performance. Integrating augmented analytics into services is an important way for transport executives to create financially sustainable customer experiences.

Mobility-as-a-service has raised community expectations for integrated trip experiences. No one accepts relying on a single-mode and then waiting for the last mile. Augmented analytics puts the customer in focus by understanding choices, linkages and available modes, to create the prescriptive option that retains patronage. By rolling up trip habits for multiple personas, the operator and funder of services can quickly establish whether surpluses are sufficient to sustain a service or journey type. Greater certainty of service ultimately benefits the customer and builds the operators reputation as a trusted brand.


Dave Bonn is Managing Director of Bonn Business Solutions.

For more information on how Silicon Billabong can help take you on the data discovery journey call Errol Kruger on +61 481 120 883 or email him at

Share now...