The Center for Data Innovation spoke with James Hodge, chief technical advisor (EMEA) of Splunk, a tech company based in San Francisco that uses software for searching, monitoring, and analyzing machine-generated big data. Hodge discussed how Splunk’s platform and use of AI technologies aim to turn data into business outcomes while benefiting society.
Eline Chivot: What is the vision Splunk’s founders had when creating the company, and what problems has it set out to solve?
James Hodge: Splunk was founded in 2003. The founders were working in IT operations, and they wanted to find something really simple to address the “error” i.e., when running systems and something goes wrong. Is it caused by the Internet connection, the Wi-Fi, the device? Is it a cybersecurity incident? They thought that there must be a way, when ingesting data, to identify and investigate that place where it went wrong, to figure out where it starts.
That investigative approach which is the foundational concept behind Splunk is very powerful because we can focus on structuring a world that is rapidly evolving all the time. At university, I was taught how to normalize a database, or how to do relational mapping—all that is important but it is a difficult process when dealing with an ever-changing, messy environment, in which there is this explosion of data and devices. Splunk’s founders wanted to allow someone dealing with such a tricky problem to be able to investigate what’s happening, right here, right now.
The hard part is to try to do it at scale, and with ease. Our approach is about being able to take any data source, any type of data, in any format, at any time scale, in real-time, into our platform to be able to use this data to answer any question, to support any decision, and to enable any action. We work with a broad range and diversity of data, for instance flight data, as we are embedded in the F35 fighter jet, or train data, data from Porsche’s electric vehicles, factory data, elevators, data from some of the biggest retail web presences in the world, and of very small companies. Our system gives us complete flexibility to take any of those different types of data from any of these sources, transfer this data into our platform, and then at the time of search, to create a schema to be able to “probe” this data—asking my question to address my problem, finding correlations, and making sense of the data.
Chivot: Are organizations aware of the value of their data? How is Splunk helping companies leverage that?
Hodge: For our Dark Data report, we recently partnered with the Enterprise Software Group (ESG), who interviewed a group of a hundred IT leaders about attitudes to data, and particularly about using data for competitive advantage. ESG identified three categories of companies based on the survey’s results. Forty percent were “data deliberators”—these aren’t really using data for competitive advantage, they’re only thinking about it. Forty eight percent were “data adopters”—they have data initiatives but they’re only starting that process to think of data as a competitive advantage. Eleven percent were “data innovators.” Beyond the fact that they drive more revenue, data innovators were two times more likely to be more successful with customers, because they understood them better. They were 10 times more likely to drive 22 percent of revenue through new product lines. What is interesting is that this category had created that data culture which involved the empowerment of those micro decision-makers, who are able to get a product to market faster. What the study also revealed is that many companies have a lot of data, but very often they are not even aware that they do, or they don’t know how to exploit it.
It’s really hard to predict what you’ll need data for in the future. As mentioned, the ability to manipulate data into the schema that you need gives you flexibility, as you are writing that search. When we ask our clients about what Splunk does for them, they often say that it is liberating: they don’t have to “pre-think” what they will need to do in the future. In my view, the most important part of a business is not just getting data to companies’ executives, but it is the staff that are on the frontline, enacting the transformation that these executives have set a vision for. These are the people making microdecisions everyday that end up having a macro impact. By giving them a very flexible data platform, we give them more of the time they need to answer that question or address that problem on the fly. This enables them to get the right data and the context to make the best-informed decision they can, and move on to the next one. That becomes liberating because you then start to understand what’s happening, and you have more confidence.
Our system also enables our clients to make predictions based on their data. We find it interesting as a prosthetic for decision-making. It is useful to understand what the goal of an AI system is and how to then interpret it, but even more relevant to have and use the raw data, drilling into it and investigating what actually happened. This is what gives the power to create a learning loop with the AI system, to enable trust of that system, to make informed decisions. Marrying up the ability to accelerate decision-making with AI and the ability to trust and understand it with the raw data and the context at hand can really empower companies to adopt AI much faster.
Chivot: Why do enterprises rely on your software? Can you give examples of how using Splunk provides value to their businesses?
Hodge: Splunk first started in the IT operations space, but we then saw our customers were also using that data for security. What that data represents is not just a piece of machine data, IT data, or security data. It is a record of an interaction that took place, and that was probably initiated by a person. So when we start to think about it that way, what we have is a history of what’s happening with the whole digital estate. I can then ask a couple of fundamental questions: Was the experience my business provided secure? Did that experience work? What did someone do with that experience? How can I learn from how people interact with my digital estate to provide an even better experience, so that I can iterate through that?
A real aha moment for our customers is when they realize that data is not about IT operations, security operations, or getting an app to market, it is a way to accelerate their transformation and business goals.
How do you empower a team? You give them data in context. If a development team is trying to get a new meaningful experience out to the market with applications in development, the first thing you need to do is to look at how you secure that. And by bringing in that perspective early on, having data as a common language for teams to start to understand different perspectives, we can give them the right guidance and guardrails, so that they can accelerate through that transformation.
In addition, we see a huge pivot to cloud, which is enabling businesses to transform much faster, but with the abundance of different services and techniques, it is adding a new complexity to system management. So for businesses, being able to, with an operational view, gain that understanding and awareness of that significant digital footprint gives them the confidence to take the next step and then another step, so that they are not running their operations blind.
Airbus has used Splunk’s platform for real-time cybersecurity monitoring and customer service. Splunk also gives Airbus visibility for their warehouse management. They deliver millions of spare parts a year within a few hours to minimize the time during which an aircraft is not flyable.
We also work with Deutsche Bahn, to help them increase transparency by getting a real-time view of their locomotive fleet. This way, they can lower maintenance costs, understand availability, reduce downtime, and curve the impact this has on customer service.
We sponsor the McLaren Formula 1 Team. It has been really good fun working with them. If you work for a retailer, you and your staff members may get a chance to wear the clothes that retailer sells, it’s relatively affordable. If you’re working for McLaren however, it’s highly unlikely that you can own a McLaren car. However, McLaren wants to bring the McLaren experience to their staff in everything they do. If you work for McLaren, this experience is embedded, for instance when you first receive your laptop, when you first log on in the morning, when you make a decision for a racing team. They have this kind of approach about data informing the performance of their cars. One of the things we worked on with them is the virtual Grand Prix series. They can’t race at the moment, so they’ve been racing the Formula 1 2019 team game on Codemasters, a publisher of racing games, and they’ve been doing these races every other weekend during the pandemic. The game produces realistic telemetry very similar to Formula 1. The driving tricks they use at the moment are about as accurate as a simulator was for a real F1 team two or three years ago, so the physics engines are pretty realistic. We get around 180 to 300 data points per unit of time about how the car is performing, so for instance, for a tire, we know what surface it’s on, we know the inside the outside pressure, the temperature, we know the X, Y, and Z forces’ movement on it, and the G force. We have analyzed these data points with the race engineering team, to check the performance of how drivers are performing in the game. McLaren invited guests to drive as part of this, including a football player. We joined a training session, and used data as part of understanding how that person was driving, and so that professional drivers and their head of race engineering were able to check when one is too heavy on the brakes here or there, or when this is how to do stirring, or which areas one should focus on first to improve one’s time, etc. These test drivers were not racing drivers, but using their data could help for an easier implementation of performance improvement with the professional drivers. That whole process was data-informed. We also produced some internal insights with two professional drivers to show who was the most consistent, who had the top speed, or who had the best lap.
McLaren did this as part of their transformation, as the results we provided have helped them showcase internally how to use data. Changing a data strategy isn’t just about whether you have the right analytics in place, whether you have use cases, but also about engagement, about enticing a cultural shift, towards adopting data and understanding what data can do, and then using data as a common language.
Companies that aim to improve both business efficiency and customer experience have to consider a comprehensive range of aspects, from super-fast Wi-Fi to clean facilities. Dubai Airports have used Splunk’s data analytics. They work like a shopping center. It’s a bit of a weird use case, but toilet flushes provide an interesting example. If one toilet stops flushing, there’s an indicator that it’s either broken or dirty; our system sends an alert, so they can deploy a cleaner immediately to fix it rather. When you start thinking about what data is—as mentioned, a record of an interaction that takes place as representing something that happens in the real world—what does that actually represent, and what would I then do about that?—this use case really highlights how much integrating such a thought process is powerful in the sense that there are many things you can do when using data for enhanced customer service.
We’ve also worked with Gatwick Airport, which has a service-level agreement (SLA) where they can get everyone through security within five minutes. For an airport, this is pretty efficient. The way they did it was by using the data we provided: Looking at the business and leading indicators, such as traffic patterns on the motorway to get to the airport. If there’s a traffic jam, the airport receives an alert, which means they will know that people will likely be arriving late to check in, so they would put their staff on a break, and then when suddenly people start arriving en masse, they would deploy them on all the check-in desks to try to get as many people through as possible.
Chivot: Your platform helps a broad diversity of industries, businesses—but also public sector organizations. How is the public sector embracing data-driven approaches?
Hodge: When it comes to the public sector, there’s a legislative, policy side to it, and there’s government as a customer, which for us represents the implementation side of it.
Governments are starting to adopt frameworks such as for the development and use of data and AI. Guidelines help companies develop a thought process about the principles of data and what to do with it—that is really encouraging because that can provide guidance to innovate and build methodologies, and that can enable more confidence in adopting technologies. The more the government works with the private sector, the easier it becomes for companies to innovate by navigating through changes and regulations. It’s also been healthy for the public sector itself, as it is starting to lead the way through its understanding of how the public and private sector can work together to achieve common goals.
In the EU, there is a gradual increase in innovation, a growing uptake in the use of AI on the market, and there are now more publicly-funded masters and PhD programs in AI. The difference between the EU and the United States however, is that the United States has had a very strong startup mentality historically. The EU is only starting to see data as providing a competitive edge. The EU sponsored this “EUvsVirus hackathon” in April, which attracted over 20,000 applicants. The use of AI was in almost all of their entries. This shows how it’s really at the forefront of European minds today.
What can be a barrier for the public sector to adopt and use data and AI is the lack of guidance on how to procure AI-based systems. We partnered with the World Economic Forum and their Center for the Fourth Industrial Revolution to publish guidance that includes a toolkit on exactly that. That is not just a technology approach, it is about policy, efficiency and risk, understanding of data, of the goals, and building a framework to implement data- and AI-based solutions.
Chivot: During the pandemic, researchers and companies have come up with solutions to support evidence-based decision-making, help save lives, reopen public spaces, and restore damaged economies. How did Splunk step in to help companies?
Hodge: What we’re seeing as a response to COVID-19 crisis is the acceleration of companies’ transformation. Business plans about ways to work and shift activities might have been a three-year process before, but now companies will want that to be a nine month- or a one year-process. When they look at technology knowledge and how to achieve that transformation, they think about adopting cloud and technologies like AI, because it has enabled people to adopt faster ways of doing business. As mentioned before, that change has brought its own complexities, for instance managing multi-cloud and hybrid cloud deployment, securing this infrastructure, and providing the right governance around it is tricky. As a result, we see a shift in the type of conversations we’re having: Clients want to ensure they have secured their cloud environment and how to do that, or to know how to get a service out as fast as possible while understanding what they’re building. In other words, there is a desire to innovate fast on the cloud but also to understand what’s the guardrails and policies they’re going to put around to address concerns over security, safety, and compliance.
With Remote Work Insights, one of our products, we helped companies shift to remote work and help their employees work from home efficiently and safely, including by publishing content to assist them such as guidance to manage Microsoft’s Teams, Slack, VPN usage, or file sharing. This helped them understand what the priorities are in keeping a business online and ensure the staff has access to everything they need.
We now work with organizations through their process of returning to work. We’re giving awareness and insights into two core functions: The chief human resource officer and the chief information officer. We bring them together by giving them data and a framework to help them understand that data better, so that they can make the appropriate decisions for their business. Not one business is the same, therefore, the way each business is going to return to work will be different for each of them. There are different offices, different work patterns, different expertise, and different locations with different situations to deal with.
At Splunk, we believe data is the key to the response to COVID-19, including about finding a cure. We’re supporting pharmaceutical companies as much as educational initiatives. For instance, with the EUvsVirus hackathon, one of our teams was placed as a finalist. Our project was about how educational authorities can support the EU and the education journey by checking how students are engaging and participating with digital learning platforms. We’re doing that for the University of Nevada as well. When students are not engaging, we can alert an educational supervisor who can proactively check on them to see if they need any extra support, if they’re having difficulty accessing the platform, etc.
What we often forget is how much the human race has advanced over the last 100 years. Over one billion people have been lifted out of poverty, and literacy rates have increased. A lot of that is due to data, more precisely about being able to share data—which is, in the end, information, knowledge, and wisdom—around the world to allow people to do things differently, approach new ideas, accelerate individual journeys, and educate people.
In the context of the recovery and digital transformation, I think there are two key responsibilities in business currently, when using data in order to restart the global economy: The primary one is towards your workforce, because the most important asset to an organization is its people. They are the ones bringing intelligence and innovation, they are the company. You need to make sure they’re safe, secure, and confident—by putting them at the forefront of your business’ vision. The second responsibility would be about going through this transformation that almost every other business needs to go through now, at a rapid rate, but with control. You can go fast if you have control and awareness of the situation around, and empower people with data in context.
