Social Data Analysis Techniques for Pandemic Response

AI in Epidemiology: The 2025 Skill Map - I spent the last 3 weeks building this no-nonsense roadmap so public health professionals, researchers, and data scientists can break into AI-powered epidemiology in 2025. Save it. Share it. Apply it ➦ 1. AI for Disease Surveillance: AI is changing how we detect, track, and forecast outbreaks in real time. To add real value here, learn to: →Build predictive pipelines with epidemiological time series (ARIMA, LSTM, Prophet) →Integrate mobility, search, and social media data for early-warning systems → Use anomaly detection for unusual patterns in case counts or symptoms → Combine mechanistic (SEIR) and machine-learning models for hybrid forecasting Tools: Python, PyTorch, Prophet, GLEAMviz, Google Health Trends API ➦ 2. NLP for Epidemiologic Intelligence: Natural language processing is revolutionizing outbreak reports, social listening, and misinformation detection. Key skills: → Text cleaning and entity recognition (diseases, symptoms, drugs, places) → Topic modeling and trend analysis for digital surveillance → Sentiment and misinformation classification → Fine-tuning domain-specific LLMs on health text (BioBERT, PubMedBERT) Tools: spaCy, Transformers, scikit-learn, LlamaIndex, LangChain ➦ 3. Explainable AI (XAI): Health data isn’t useful if policymakers can’t trust it. Epidemiologists need to master transparent AI methods. Learn to: →Interpret models with SHAP and LIME → Identify feature importance and bias in prediction models → Build dashboards that visualize explainability for policy stakeholders → Audit algorithms for fairness and equity Tools: SHAP, LIME, ELI5, Plotly Dash, Streamlit, Tableau ➦ 4. Spatiotemporal Modeling Most public health questions are “where and when.” AI brings precision to those answers. Skills to focus on: →Build spatial regressions and GWR models → Detect hot-spots using LISA or Moran’s I → Combine satellite, environmental, and social data for risk prediction → Train spatial ML models (XGBoost + spatial lags, graph neural networks) Tools: GeoPandas, PySAL, XGBoost, Kepler.gl, ArcGIS Pro, QGIS ➦ 5. Causal AI for Policy Impact Prediction shows what — causality shows why. You’ll need to know: → Difference-in-Differences and ATT(g,t) estimation → Structural causal models and DAGs → Counterfactual prediction using machine learning (causal forests, DoWhy) → Policy simulation with synthetic controls Tools: DoWhy, EconML, PyMC, CausalImpact, R (did, fixest, causalforest) If you work in public health, medicine, or data science - this is your AI decade. Don’t wait for “AI experts” to explain your field. 💡Key takeaway: "Epidemiologists who understand both data and disease will define the next generation of health intelligence." If this roadmap helps you see where AI meets epidemiology - hit save, share, or tag someone building the future of public health. #DigitalEpidemiology #PublicHealthAI #Epidemiology #DataScience #AIEthics #HealthInnovation #AI

This new paper by Sergio Consoli et al explores how generative AI can transform unstructured outbreak data into structured, searchable knowledge. The team developed an epidemiological knowledge graph (eKG) using WHO Disease Outbreak News (DONs), applying an ensemble of large language models to extract details such as disease name, country, date, and number of cases or deaths. The researchers used open-source models including Mistral, Zephyr, and Meta-Llama to extract information from over 3,000 outbreak reports. They structured this data into a FAIR-compliant knowledge graph, linking it with biomedical and geographic ontologies. The resulting resource—comprising nearly 3,000 outbreak events—is now publicly accessible via SPARQL endpoints and visualization tools. This matters because many official outbreak reports remain locked in prose, making them difficult to analyze at scale. With eKG, public health professionals can conduct detailed, structured queries across decades of global outbreak data. This significantly improves our ability to track, analyze, and respond to emerging health threats. The big takeaway: AI can unlock the full value of legacy outbreak data by transforming it into structured, interoperable formats that support real-time analysis and response. This approach opens new possibilities for integrating informal sources like news and social media into formal disease surveillance systems, advancing global preparedness and early warning capabilities. https://lnkd.in/ePc54yvQ #GlobalHealth #PathogenSurveillance #HealthInnovation #PublicHealth

I am pleased to share our study, 'Using Social Media Analytics to Understand the Impact of Government Intervention on Consumer Behaviour During a Pandemic,' via the Australasian Marketing Journal. The study aims to examine the impact of contagious disease cues and the government’s nonpharmaceutical interventions on customer sentiment related to perishable products with high touch frequency such as fresh fruits during a pandemic. 🍇 🍈 🍉 🍌 🥑 🍎 At the time of submission, the team consisted of Sara Quach (Griffith University), Alec Zuo (University of Adelaide), Park Thaichon (University of Southern Queensland), Robin Roberts (Griffith University), and Wenzhu Tang (University of Adelaide). Through social media analytics 659,537 individual tweets were collected from Twitter, by users based in two major cities in Australia (one of two major cities is the world’s longest COVID-19 lockdown). Our findings suggest that in general contagious disease cues will negatively affect consumer sentiment about fresh produce, and this effect is moderated by government interventions. In particular, except for customer sentiment related to the use of chemicals, daily case numbers had a significant negative impact on customer sentiment related to freshness, healthiness and contamination in both Sydney and Melbourne. The findings offer significant implications for marketers and policymakers to effectively adapt to emerging customer demands and expectations. The results also confirm the effectiveness of non-pharmaceutical interventions, specifically lockdowns, in mitigating the negative effect of contagious disease cues on customer sentiment. Link to the article: https://lnkd.in/gwqtJmp4 We would like to take the opportunity to thank the reviewers and editors for their comments and suggestions. Liem Ngo Funding The author(s) received financial support from the Australian Centre for International Agriculture Research (ACIAR). Title: Improving smallholder farmer incomes through strategic market development in mango supply chains in Southern Vietnam (AGB2012061). Date received: 16 September 2022; accepted: 12 June 2024 #ConsumerBehaviour #Pandemic #freshproduce

➡️ New Pre-Print Paper (Arxiv): 📄 Non-traditional data in pandemic preparedness and response: identifying and addressing first and last-mile challenges ✍️ by Mattia Mazzoli, Irma Varela-Lasheras, Sónia Namorado, Constantino Pereira Caetano, Andreia Leite, Lisa Hermans, Niel Hens, Polen Türkmen, Kyriaki Kalimeri, Leo Ferres, Ciro Cattuto, Daniela Paolotti, and Stefaan Verhulst 🔗 Read it here: https://lnkd.in/eXXe2tGX 🤔 The COVID-19 pandemic served as a global stress test for how we integrate non-traditional data (NTD)—from mobility traces and social media activity to wearable data—into public health decision-making. ➡️ As part of the ESCAPE project, and drawing on an expert workshop (Brussels, 2024) and a survey of European modelers, our paper assesses both the promise and persistent limitations of NTD in pandemic preparedness and response. We distinguish between: 🧭 “First-mile” challenges — accessing, harmonizing, and standardizing data. 🚦 “Last-mile” challenges — translating insights into timely, actionable decisions. Some Key findings include: - 66% of datasets faced access issues; - Data-sharing reluctance for NTD was twice that of traditional data (30% vs. 15%); - Only 10% of respondents could use all the data they needed. 🤔 Barriers extend beyond the technical—encompassing institutional inertia and weak data-to-policy translation. To move forward, we propose a roadmap that emphasizes (among other things): ✅ Legal and technical frameworks for data access; ✅ Fusion Centers and Decision Accelerator Labs to bridge analysis and action; ✅ Networks of Scientific Ambassadors to connect scientists and policymakers; ✅ A shift toward a culture of data solidarity and sustained institutional readiness ✅ Advancing data stewardship. ➡️ Grounded in the lessons of COVID-19, this work aims to guide the responsible use of non-traditional data not only for pandemics, but also for emerging challenges like climate shocks and humanitarian crises. 💻 Learn more about ESCAPE: https://lnkd.in/eTU5-7DD ISI Foundation, The Data Tank #DataForHealth #Pandemic #NonTraditionalData #DataStewardship #DataSolidarity #PublicHealth #AIForGood #EvidenceBasedPolicy #DataGovernance

🔬 Molecular surveillance is the gold standard for tracking diseases. However, as we exit the pandemic era and funding is cut, the world needs scalable, alternative methods. Surprisingly, one of the most promising tools might be... Elon Musk (i.e. Twitter, i.e., X). 🐦 𝗛𝗼𝘄 𝗧𝘄𝗶𝘁𝘁𝗲𝗿 𝗣𝗿𝗲𝗱𝗶𝗰𝘁𝘀 𝗙𝗹𝘂 𝗢𝘂𝘁𝗯𝗿𝗲𝗮𝗸𝘀: Every day, millions tweet about their health. These tweets, when analyzed, can provide insights into potential disease outbreaks. 📊 A recent preprint by David Martín-Corral et al. (https://lnkd.in/gdximJT8) focused on Twitter messages in Spain about flu-like illness and analyzed different classes of users for their suitability as "canaries." They proposed a unique "sensor" approach, using highly active Twitter users as early warning systems. Their findings improve on earlier social media-focused approaches and identify specific classes of users that could predict actual flu cases up to four weeks in advance. On the one hand, this is nothing new: 📚 𝗛𝗶𝘀𝘁𝗼𝗿𝗶𝗰𝗮𝗹 𝗖𝗼𝗻𝘁𝗲𝘅𝘁 𝗳𝗿𝗼𝗺 𝗟𝗶𝘁𝗲𝗿𝗮𝘁𝘂𝗿𝗲 - Twitter Improves Influenza Forecasting: https://lnkd.in/gvtKUQmT - National and Local Influenza Surveillance through Twitter https://lnkd.in/g_-iX-kT However, it's good to see this area is still being poked at to be improved further. It's weird and not what we normally think of for disease surveillance. But, there's a lot of data out there, and this is one interesting source that probably isn't taken as seriously as it should be. 🔮 𝗧𝗵𝗲 𝗙𝘂𝘁𝘂𝗿𝗲: While not everyone tweets, and not every tweet is accurate, the potential of this method is undeniable. As research progresses, we might soon receive early warnings about potential disease outbreaks, all powered by social media insights. I expect this will all seem obvious to our future AI overlords, but combining meta data with biochemistry has a lot of potential. #HealthTech #DigitalHealth #Innovation #PublicHealth