GRIFFITH UNIVERSITY - RELATIONAL INSIGHTS DATA LAB

EPISODE TRANSCRIPT

Rhetta Chappell (host): Hi, and welcome to Show Me the Data, a podcast where we discuss evidence-based decision making and the ways in which our lives interact with and create data. I’m Rhetta, your host for today, and I’m a Data Scientist at Griffith University. Show Me the Data acknowledges the Jagera peoples who are the traditional custodians of the land on which we are recording today. And we pay respect to the elder’s past, present and emerging.

Hello, hello, and welcome to Show Me the Data. Today we have a very special guest in studio, which of course, all of our guests are. It is my pleasure to introduce you to Dan O’Halloran, a man who wears many impressive hats and has held many critical senior executive positions in both public and private health. Where we first connected and had the pleasure of working with Dan was when he was the Senior Director for System Performance and Senior Director for COVID-19 Analytics for Queensland Health during the pandemic. Since 2020, we’ve had the pleasure of collaborating with Dan on an all of government data ecosystem pilot and a number of other fascinating and impactful projects. Also joining me today is my co-host, Tom Verhelst. Dr. Tom Verhelst is the director of the Relational Insights Data Lab and the Griffith Data Trust, which is also where I work, Tom and I could have honestly spoken to Dan for hours, and we nearly did, because well you’ll hear, Dan is a fount of knowledge and as a result, we didn’t want to cut out any of his rich insights. We’ve packaged this conversation up into two sections. Please enjoy part one. And we encourage you to come right back here and check out part two afterwards.

All right. Well, thank you so much for being here today, Dan. And you as well, Tom, I wanted to start by asking you about the importance of data storytelling and contextualization. I read recently that around 90% of the world’s data was created in the last two years, and that by the beginning of 2020, there’s 40 times more data present in in the digital realm than there is observable stars in the universe. I just thought that was a really kind of cool and interesting visual to kind of think about the vastness of that. And then the potential of this data is obviously huge. But we do know that both governments and kind of companies and industries are struggling to turn this data, and this kind of huge amount of data into useful and actionable insights. Could you please speak to how you go about finding and communicating these kinds of needles in a haystack? Or how do you deal with this kind of huge amount of data and doing something useful with it?

Dan O’Halloran: Great. Thanks, Rhetta, that’s a really good question. There is, I think, probably about five or 10 years ago at a CPA conference, a lot of discussion was around the wealth of data and so it has expanded, exponentially increased. And we start measuring and capturing data and a whole lot of things we never used to. And so, one of the key things that data has, is that it can actually allow us to tell the story of people with how they experience public services. And so, whilst we can go out and ask people about their experience, from a survey, or through interview or chat, the actual data represents the journey that a patient or a citizen actually takes an in engaging with a service or a sector. And it’s then on those of us within those roles to actually then give life to that story, or the aggregate of that story. And so, one of the challenges that governments often have is that, do you actually see a full picture of the citizens experience and so? Or are you only seeing it from a perspective of say, the people entering the data or from the clinician? And so, if we think about data within a lot of health departments, at the moment is that a lot of that data is based on diagnosis, which is what their clinician actually said was the presenting or what the problem was, and but there’s a lot of evidence that was done by say, John Hopkins, about a decade ago, that when they looked at over a 25 year period, most of the challenges and was with misdiagnosis, so most patient safety issues with misdiagnosis. So the question then is as well, is the data actually able to tell you what it is that you want to tell? And the other real important thing that data adds value for is that it allows you to look at a situation relative to others and so relativity. And allows you to then contextualise that with a point in time, allows you to see things that happen or tell a story over time. Does the data focus around a transaction? Or does it focus around the citizen on the patient? And so does it actually allow you to look at, okay, they were seen in time? But is that the outcome that you’re trying to achieve? The question sort of is, so what?

With that, it allows you to say, well, with the data, you can actually run counterfactual arguments, which you can’t do without data. Really, you can have a sort of conceptual conversation and there can be people present different views and arguments, but often that’s conjecture. And so, but the value by having different data, you can actually start running counterfactuals, you can start seeing relationships across different aspects. And so, one of the key things around and insights that we historically have done with teams that I’ve led, is looking at the relationship between access, then that relationship with safety and quality and that relationship then with finance. And so, what often happens is, you’ll hear stories from people who just can’t access the system, or I had a really bad experience. But putting that into context as well, what’s the causation of that problem? And one of the big values that data then brings, is that you can then contextualise that to those communities, the problems not the same in every community. And so, what you could do is if you’re leading a state, or you’re leading a State Health System, or a national health system, just having conversations with a few people on what they think the issues are. Those issues that they’re presenting may be very real, they’re real, and the context upon which they operate and where they work, that may not necessarily be the case in all communities. And so, I think when we’re looking at insights, it’s around well, unless you contextualise and start looking at causation, then you can’t really then start understanding or having very good policy.

Rhetta: Yeah, I think that’s really interesting. And I think from that kind of place-based kind of robust view of what somebody actually looks like, because none of us exist in vacuums and we don’t kind of isolate services or anything, kind of an isolated ways or in that sense, so I really liked that you touched on that. And I think with that counterfactual, and kind of evaluating the trade-offs, or kind of different scenarios, do you think that’s something we’re doing well in Australia? Or we’re doing enough of? Or something we can improve on?

Dan: Oh well, a loaded question. No matter how good you are at something, you can always do better. And so and the reality is, that no matter what you do, sometimes it will never be enough. Because it depends actually on the situation that you’re in. And so, if I take myself back, when we were grappling with how do you prepare for the COVID-19 pandemic, then how much is actually enough? How much counterfactual do you actually need to have to then actually influence change? And so, I think putting that into context to the situation that you’re in, when you’re doing that is actually really important and what the ramifications are. And so, conceptually, what I would do is probably put it on a matrix. It is that for the implication of the decision, and say, how much of an impact that’s going to have on society? And then what the potential trade off of that impact is? is that those that then sit within the top right hand corner is where we should be doing counterfactual arguments. And do I think we’re doing enough in that? No I don’t because we have today, the largest number of doctors and nurses employed in Australia, in our healthcare system, we have records amount of funding employed in that system and the first time in my living memory, we’ve heard reports over the last 12 to 18 months of people dying when they’re being ramped by an ambulance. And so, And the sad part about that, and the concerning part about that is that those people on that ramp had no choice. And so, they thought they did the right thing, they called up, they got the ambulance, I thought that they were safe within that system, unfortunately, that those people may have subsequently died. But the fact is, no one wants to die on a ramp, I don’t think that has any level of dignity. And so do I think we’re doing enough and counterfactual then I don’t think we are. Because if we were I don’t think we would actually have some of the situations that we face today. Because the situations that we face today are extremely challenging and I actually don’t think we really know. There’ll be people say, or well I know what the problem is, but to truly know what the problem is, in every community, I actually don’t think anyone can say and put the hand on their heart and actually say that, because they don’t have the data. And so, unless you have data that looks at the whole journey of the patient, and we can contextualise that by community, then we should actually be identifying what these issues are by community.

Tom: Based on what you were just saying that we have a challenging healthcare system. What roles do you think data could play for both market and non-market responses to that sort of public health policy? Because do you think there’s this solution there are not?

Dan: There’s always a solution.

Tom: Positive.

Dan: Let’s accept that. Like if you get to the point that you say that there’s no solution that you can see. And so, if any leader, or any decision makers at that point in time, now think they should be making the decision to step back and let someone else actually step up. Because the reality is, is the rest of us expect someone to actually look for or hunt for a solution. So, with these solutions, so there can always be a solution, I think there always should be. And we should always hunt and find a solution. Those solutions could be market, and they could be non-market. And so, what I mean by that is that in a market, people have choice and that means there is information within that market that addresses asymmetry of information. So if we think about the healthcare system, that means is information for consumers to understand risks that is not then provided by clinician as provided by the market, which is the same as what you do for the ASX or any other major sort of decision that you’d be making that could impact you for life. You then would make a choice as to which provider would you then go to and you’ve then accepted that risk. And so that’s a market to me, that’s a market sort of option. And so, if I think about that, with the current structure of the Australian healthcare system, is that Medicare enables a market. And so that enables Australians to have choice on which GP they would like to go to. Now, that doesn’t mean that they can access it, that’s a different issue. And so, but it does provide them choice on where they would like to go when and how they would like to experience that system. But I think about a non-market situation, a non-market is when you don’t have a choice. And so, at the moment, what is a non-market example in the Australian healthcare system? That would be emergency departments and so at the moment, there is no choice for emergency or urgent care, other than to go to a state-based emergency department system. And so, what is the role of data in a non-market environment? Data provides transparency, and it provides accountability. If you choose to have non-market dynamics, and you don’t have transparency on what is actually happening within those services across the entire federation, then how do you know that what’s happening across that entire federation is equal or, more importantly, equitable. And so otherwise, it’s just someone’s view, which in my view, is just conjecture. And until you actually start, and there is no one in this country that works in every single emergency department in this country. So that doesn’t discount the stories that people say that they’re under pressure, that they need more resources, that may be true in their department, but it doesn’t necessarily mean that that’s true everywhere and it also doesn’t then go to the point around contextualising that experience, as to what is the cause of that. Because in some communities, the cause of that issue could be is that the GP left that community, the only GP in that community left that community. In another situation, it could be there is that there is access issues to get getting patients admitted. The other issue is it could be a public health care crisis, like what happened in Wuhan. And so, I think data has a significant role to play.

Tom: It’s kind of very interesting how sort of the market versus non-market, it leads to vastly different outcomes. But the challenge frequently is that we sort of set this, this is set at a government level and whether something is optimal, a market or non-market, highly depends on the context of the community, of the technology, of the time we’re in. And, you know, if there’s if there’s a pandemic or not, while the context changes so quickly, you can’t switch between market and non-market quickly. I think that’s why you sort of see, like in the US up till about maybe 30 years ago, health was really good, health care, as we didn’t get it privatised a bit more. But if you look at it, now, they spent, like, three times the amount of German spend or something and their outcomes are not as good.

Dan: Just on that, though, is that we have to be really careful about what we deduce or conclude with aggregated information.

Tom: That’s true, because for some people, it’s better.

Dan: Yeah. yeah. And so, in every system, there will always be exemplars, and there will always be bad things happening. Let’s just accepted that right, and so but the fact is, is that looking at aggregate data doesn’t actually really give you an understanding what’s actually happening. And so, what I mean by that is that if you’re living in Smithton, Tasmania, or you’re living in Straughn, does it matter what’s happening at an aggregate level for Tasmania? No, all you want to know is for the people that I love, and then people that that are that mean something to me? Can they access care in my community when they need it?

Rhetta: And this kind of place-based approach is something we do at RIDL and is provided a lot of value to the different communities that we work with and we keep it as small as it can be. Like an SA2 and we’re kind of unpacking all the data within the SA2, whereas it can be as big as a Queensland, but kind of using the data at different levels of granularity based on the project there. Touching on what you were saying, with regards to regional health in the market and non-market policy responses, is there an asymmetry in terms of if you’re in a regional area are you faced with more non-market and potentially, then less transparency? Or is it that not something that affects like from a regional kind of urban perspective, more like, I’m wondering if you’re saying with that example of the only GP leaves, then you kind of left with this, you only can have that non-market? Is that something that regional policymakers and consumers are facing more, I guess?

Dan: Yeah. So it’s interesting. There’s one point I just want to make for Tom, is that there, in the US system, there are some significant exemplars, but doesn’t mean everyone has access to it. And so, the US if you, one of your earlier questions Rhetta was around what countries are doing really well? And so, I think there are examples in the United States where data is being used really well and an example of that is, is where data is being used within inpatient settings through electronic medical records to identify all cause harm in real time. And so, if I contextualise, that is that incident reporting systems for patient safety, some evidence some time ago indicated that only 3 to 5% of all harm is captured by those incident reporting systems. And Don Berwick, the father, or founding father and patient safety, wrote an editorial earlier this year in January in Health Affairs, they’ve basically calling out, that with the incident data, what’s the point of actually doing rates? Now, if that’s the case, then there’s no point doing comparisons, which means that if you’re running an aged care provider, or you’re running a healthcare system, your ability to understand relative risk across that as an executive, you can’t do that with that data. And so, so there are advancements in the US that have been looking at inpatient data to actually address things and risks much earlier. Moving to the question around regionality and metropolitan, one of the biggest lessons that I learned from LSE is that it always depends, always depends, and it depends on the social contract within that community, it depends on the economic situation within that community, it also depends on how many players are in that community, and then also demand, right. And so, you then have the relationship between supply and demand. And so, if we think about market dynamics, and markets will always fail, or are likely to always fail in smaller markets, because there’s less competition, right. But the other thing is, if there was competition in those smaller markets, there wouldn’t be enough margin to then actually make it financially viable. And so there actually has to be some understanding when we’re creating a market, is well, what is the definition the market? That’s a human based design, right? And so, on that basis is that it’s it is by your own definition, and what that is, and so, you could create a geography that creates a market that is actually sustainable. But what ends up happening is that often is the case is that that thought isn’t actually localised or contextualised to a place-based commissioning model. And so, as an example, you it would be very hard to transition a non-market provider. Okay. So services that used to be provided for say NDIS, so disability, aged care to a market. Very, very difficult for regional and remote communities. Because the reason why often the case is that you had government or say non-for-profit providers operating that market, is there is no margin. And so it becomes a social good that’s then provided. So, markets often are more likely to be successful, and provide that competition and transparency in much larger economies.

Rhetta: Oh, that makes sense. Yeah, that’s good. Let’s just switch questions. I was listening to Freakonomics podcast the other day, and they were doing a replay episode on hospital rankings, I think it’s the US News and World Report. They rank all the hospitals every single year. And obviously, depending on where your hospital probably falls on that list, you either love or hate the rankings? Do we have anything like that in Australia? Or should we? Is that something we should be striving for?

Dan: It’s a low question, because I was privileged enough to actually be employed by the National Health performance authority when it existed a decade ago.

Rhetta: Oh, so it doesn’t exist anymore?

Dan: It doesn’t exist anymore. And so the National Health performance authority was established under the 2011, I believe, national health reform agreement. And that was to provide transparency in a non-market environment. And so, the intention of that was to actually start understanding variation. And it was actually the first time in Australia’s history, that the public actually started to have access to variation across communities in our country on what it was actually happening. And so doesn’t mean everything that was achieved by the authority was perfect, no, nothing ever is by any, any organisation. And so, but what I mean by that is there well, there was limitations with that, but it was limited by the data that was available, not by the people. It wasn’t a limitation of will, it was a limitation of what data was available. And so, it reported infection rates variation, and it reported vaccination rate variation. And actually, it led to a data driven evidence-based policy of the no jab, no play policy.

Rhetta: Okay. Yes, yeah.

Dan: And it gave for the first time, insights to technical efficiency variation across the country. But the thing is, what I’ve learned since during and leading those reports, and technical efficiency variation, is that a lot of people or a lot of actors in the system, took those reports and just drove down cost. That wasn’t the intention of those reports. And so, one of the biggest learnings that I then had, having the privilege to serve in the Queensland Government as an executive was actually looking at trade-offs within the system and looking at the relationship between safety, quality, access, and finance. Because some of the finance challenges can be related to safety and quality, because people stay longer and that can then lead to access issues. And so, but that then goes to the point around, sometimes you actually need to cost more in some markets to actually deliver value. Because sometimes it’s very easy to deliver low cost, low value care. And so, whilst we want to actually ensure that, that we’re getting value for money, that is different to delivering value, because what is often people think value for money is just the lowest cost, but you want to know what it is that you’re actually getting for it. What I will just say before we move on, is that one of the challenges we have in Australia is that all of our data that we look at is primarily based around diagnoses, or the diagnosis that was done by the provider. And so, if you, if I go back to John Hopkins report, they indicated looking over a 25-year period, decade goes, one of the biggest safety issues is with that diagnosis data. And so maybe I just asked the question, maybe we should be actually capturing information on the presenting complaint. Because if we actually flipped that, and captured that in the data, and we understood why you as a citizen kept coming back to a service, we might actually start finding that your presenting complaint is the same over the last month, and you’ve come to us five times and it hasn’t been resolved. But you know what, if you look at the diagnosis data, you got three or two diagnosis that was all the same. But it doesn’t tell you that something’s wrong with that person, and they’re not actually getting what they think they need out of the system.

Tom: So, is that what you think the biggest data gap that we have in our system?

Dan: I do, because do I think it’s the biggest? I think it’s one of the biggest and so because if you are only looking at the episode of care, which is the diagnosis and what you did, you’re not actually then getting any insights as to why the person was there in the first place. Okay, let’s maybe make it simpler. If someone had a broken arm, tick, that’s pretty obvious, right? That is that, that’s the reason why they were there and the diagnosis. But that’s, they’re not the problems that the health system is facing a lot of the health system, the challenges that it’s facing us with chronic disease, and then medical diagnosis, they’re not physical or structural diagnosis, which means is, there’s more. That’s more difficult and more challenging.

Tom: Yeah, and they keep coming back.

Dan: So the question is, is that why? Right, it’s a little bit like, and I don’t mean this to be flippant, but if you purchased a car, and you thought something was wrong with it, you’d keep taking it back to the mechanic, right? And so in a in a market environment, you could go to a different mechanic, and they’d be like, okay, I wasn’t happy with the service I got I take it somewhere else, they fix it. But in a non-market environment, if I’ve got to go back to the same person every time, where else do I go? And then who else in the system is looking at the data to identify that there’s a problem in that non-market?

Rhetta: Hi, me again, quickly, just a friendly reminder that the second part of this conversation will be continued in our next episode, please find part two where you found part one, thank you.

To listen to more episodes of Show Me the Data, head to your favourite podcast provider, or visit our website, RIDL.com.au, and look for the podcast. We hope that by sharing these conversations about data and evidence-based decision making, we can help to inform a more inclusive, ethical and forward-thinking future. Making data matter is what we’re all about. And we’d love to hear why data matters to you. To get in touch. You can tweet us @G_RIDL. Send us an email or if you prefer, just send us a letter by carrier pigeon. Thank you for listening, and that’s it till next time, take care and stay safe.