Leveraging Open Data in Scientific Research

Let’s talk about open science. A scientific breakthrough reaches its full potential when it empowers the broad community to replicate and expand upon findings. Over the last decade, we have developed and maintained several key technologies and datasets empowering an active ecosystem of more than 250,000 researchers and developers worldwide. Our commitment to responsible open science spans several critical frontiers: 🧬 Genomics: Our deep learning tools—DeepVariant, DeepConsensus, and DeepPolisher—have enabled the global community to process the exomes and genomes of 2.5 million individuals. 🧠 Neuroscience: We provide technologies like Neuroglancer and TensorStore alongside massive public datasets, such as the 1.4 petabyte H01 sample of human brain tissue. 🌍 Earth AI:  We are providing critical data for climate adaptation and urban development, including 1.8 billion detections in Open Buildings and the NeuralGCM atmospheric model. We’ve also released the Groundsource dataset for urban flash floods, comprising 2.6 million historical flood events derived using Gemini on 20 years of public data and spanning more than 150 countries. And Caravan: is a community-driven dataset for large-sample hydrology. 🏥 Healthcare: Our Health AI Developer Foundations (HAI-DEF) and Open Health Stack are democratizing medical AI, with deployments in over 10 countries reaching 65 million beneficiaries. 🦒 Biodiversity: SpeciesNet, a global-scale model, is currently classifying 2,498 animal categories to support conservation efforts. The real-world impact is the ultimate goal. From predicting floods in 150 countries covering 2B people, to predicting monsoons for 38 million farmers in India to preserving the critically endangered kākāpō in New Zealand, open resources allow for faster research and deployment As we transition into the era of AI-enabled science, we believe agentic workflows will allow scientists to transform complex methodologies into accessible, scalable tools. In this fast-paced new paradigm, communication and collaboration are more critical than ever. Open-source software and open datasets serve as the essential foundation for this ecosystem, enabling faster innovation and universal sharing of scientific knowledge. Read about real world impact powered by open science in today’s blog by our science team: goo.gle/4dnd9hx

🌍 Open Data & Open Code in Forest Remote Sensing – A Must-Have? More and more, I hear people within the forest remote sensing community open to sharing data and code – and this is an excellent development that puts us on the right side of history. 🚀 Here’s why this matters: 🔑 Consistent Benchmarking – something we haven’t been used to. Open data enables robust, efficient comparisons without excessive replication, letting us focus on understanding what works (and what doesn’t). ⚡ Accelerated Method Development – with data freely available and without strings attached, researchers can readily develop and deploy new models. 🌏 Democratization & Inclusivity – many researchers worldwide lack access to expensive sensors or the resources to collect and label high-quality data. Open datasets level the playing field, increasing the number of potential contributors and, in turn, potential solutions to shared problems. 💡 This raises an important question: is open data a nice-to-have, or a must-have for science? In my view, open data (and open code!) is essential. Not only does it accelerate progress — something we urgently need in a world dominated by a few large tech players — but it also ensures method reproducibility, a cornerstone of science🧪 Having spent some years around the computer vision community — where open datasets and code are the norm — I’m convinced that science (especially when publicly funded) depends on openness. Without it, we risk jeopardizing the very foundation of scientific progress. 🙌 To those still hesitant: think long term. Yes, someone might publish a paper with your data without you as a co-author, but you’ll likely get answers to your questions faster than you could on your own. Personally, I get far more satisfaction seeing how many people download and use our open datasets than from the one or two papers I might squeeze out of them. And in my experience, when you give to the community, the community gives back. 🤝 It’s an exciting time — let’s keep pushing forward and join the #OpenData #OpenCode revolution! 🔓📊

Data are infrastructure. Though invisible to most, I think of data infrastructure as essential to a functioning society as our highways or electrical grid. For complex reasons, a lot of data are privately owned or otherwise inaccessible. Imagine if that were true for other infrastructure projects, and how challenging progress would be. Ideally, data would be openly available for researchers to use and make new discoveries that can improve the health and wellbeing of the nation. One source all researchers should know: data collected through @RWJF research grants are publicly available. The good people at the @Health and Medical Care Archives (HMCA for short) help researchers archive datasets and make them available to others. The data can be used for secondary analysis, replication, or new studies. HMCA’s website has data on a range of topics, from food borne illnesses to housing policy to community development. There's a lot there for researchers, community groups, and policymakers alike. These datasets are not an old archive; they are living resources for answering new and pressing questions that the original researchers may have never imagined. I encourage you to bookmark this post and come back to the HMCA often in your work. And please share with your networks. The purpose of research isn’t simply to publish, it’s to put knowledge to work. This is one more way we can build on existing knowledge to improve society. #OpenScience #OpenData #Research #DataforGood https://lnkd.in/gQsQstiy

Masterclass example of reproducible research on air pollution & mortality A recent publication of the #EHMLab at London School of Hygiene and Tropical Medicine, U. of London in EPIDEMIOLOGY investigates the long-term effects of time-varying PM2.5 exposure on mortality using data from the UK Biobank cohort. We used novel methods leveraging the detailed reconstruction of individual-level exposure histories and socioeconomic/lifestyle factors. Key focus on reproducibility: we generated a synthetic dataset for 502,360 participants and released in full the R code for a simplified tutorial and the original analysis. Links: Full-text article: https://lnkd.in/dumB_dTZ Synthetic data: https://lnkd.in/dytXhsaJ R code: https://lnkd.in/dmrPbQQt These resources can be used for replication, education, or further methodological development to advance open & reproducible environmental health research #AirPollution #Epidemiology #Reproducibility #OpenScience #PublicHealth #UKBiobank #DataScience

GIS Students - If you want your work to be used, make it reproducible! And of course, if you want it on GitHub, then do it well!! Some tips to help you. 1/. Share complete projects - include data sources, scripts, styles, and outputs. 2/. Document the workflow - step-by-step instructions from raw data to final map/analysis. 3/. Specify software & versions - GIS platform, libraries, projections, and dependencies. 4/. Use open formats - prefer GeoPackage, GeoJSON, CSV over proprietary or fragmented files. 5/. Include metadata - coordinate system, units, data date, source, and limitations. 6/. Provide sample data or links - avoid broken paths or inaccessible local drives. 7/. Automate where possible - Python, ModelBuilder, or scripts instead of manual steps. 8/. Avoid “mystery layers” - every layer should have a clear purpose and origin. 9/. Organise your folders - clean structure (data / scripts / outputs / docs). 10/. Test replication - run the project from scratch before sharing. 11/. Add a README - explain what the project does and how others can run it. 12/. Use version control - GitHub/GitLab to track changes and collaborate. 13/. Respect licensing - clarify what others can reuse and how to cite. 14/. Avoid hard-coded paths - use relative paths for portability. 15/. Keep it simple - reproducibility matters more than complexity. 🛑Note: Open, clear, and reproducible workflow is how you stay relevant and trusted in the community. 🔄

Excited to share about the World Bank’s Reproducible Research Initiative in the Berkeley Initiative for Transparency in the Social Sciences (BITSS) blog! Led by the Development Impact Analytics team, this effort strengthens transparency and credibility in development economics by ensuring that empirical Working Papers include a reproducibility package, complete with data, code, and documentation. All verified before publication. With over 300 packages already published on the Reproducible Research Repository (https://lnkd.in/eGaY8AYm), the initiative is making research at the Bank more accessible, transparent, and credible. Reproducibility isn’t just about rigor, it’s about collaboration. By opening our data and methods, we invite researchers, policymakers, and practitioners everywhere to build on this work, test new ideas, and accelerate collective learning for development impact. Proud to be part of a team advancing a global culture of transparency and shared evidence in development research. Read the full post here: https://lnkd.in/efzANMC6

Plant breeding generates massive datasets, but most remain locked in silos due to trust barriers and technical incompatibilities that slow innovation. Our new study introduces "Data cohorts" (structured packages of interoperable breeding data) paired with a Data Trustee Platform that enables federated sharing while protecting proprietary interests. By implementing FAIR principles from the start and using dynamic licensing with secure analysis environments, we show how genomic prediction accuracy doubled when aggregating previously isolated datasets. The key? Standardized metadata, common genotype references, and quality metrics that let breeders access relevant data without compromising competitive advantage. When data flows freely but safely, everyone's breeding programs get stronger and innovation accelerates. https://lnkd.in/dq2dJUHk

We have pretty good chemical structure-property datasets for ML development and benchmarking, but much less so for chemical reactions. Molecular ML has benefited from curated experimental datasets mined from the literature. Resources like ChEMBL for bioactivity and ESOL for solubility normalize single molecules against well-defined experimental endpoints, while frameworks like MoleculeNet enable clean targets, consistent representations, and reproducible evaluations. Reaction data is a different story. Reactions involve multiple molecules, roles, stoichiometry, and transformations that are often incompletely or inconsistently reported. Small preprocessing choices can quietly propagate through the modeling pipeline and distort evaluation. Still, meaningful progress is being made. Dataset efforts like the Open Reaction Database and ORDerly are improving standardization and data hygiene, while evaluation frameworks such as Syntheseus and PaRoutes are bringing much-needed rigor to how reaction models and planners are compared. #SynRXN builds on these efforts by curating existing public reaction data (primarily USPTO and academic benchmark sets) into a unified benchmarking framework, treating computer-aided synthesis planning (CASP) as a collection of explicitly defined, benchmarkable tasks rather than a single end-to-end problem. Its key ideas include: 🔹Decomposing CASP into clearly scoped task families, from reaction rebalancing to synthesis planning 🔹Reserving rebalancing and atom-mapping benchmarks for evaluation only, preventing training contamination 🔹Using deterministic, leakage-aware splitting instead of ad hoc train-test partitions 🔹Tracking provenance, licensing, and versions so datasets are reproducible over time 🔹Defining standard metrics and reference baselines for consistent comparison across papers Looking ahead, progress in reaction modeling is likely to come from the gradual buildup of shared datasets, tools, and evaluation standards. As these pieces accumulate, they form an ecosystem that supports more reliable comparisons, clearer ablations, and results that can compound over time rather than resetting with each new paper. 📄 SynRXN: An Open Benchmark and Curated Dataset for Computational Reaction Modeling, arXiv, January 5, 2026 🔗 https://lnkd.in/eG8vYHzR

We're releasing RoP v2026.04: 1.33 million harmonized biomedical Common Data Elements - open for the research community. What & Why RoP pairs CDEs with semantic embeddings, value sets, and governance parameters for true multi-cohort interoperability. It's the foundation for our HiTL automated data and schema harmonization registry. Today, multi-cohort studies spend months manually mapping variables. One calls it "MoCA Total Score," Two calls it MOCATOTS, Three calls it MOCA. Same assessment, zero compatibility. Researchers spend 40-70% of their time on data wrangling (Kaggle). RoP + The Forge cut standardization time and cost by an order of magnitude. Let's make research as FAIR as possible. RoP covers every layer of biomedical data: identity, clinical , genomics, imaging, governance, and findability. Built on OMOP, LOINC, ICD-10, HPO, Mondo, and 9 other major vocabularies. Running in production across hundreds of thousands of samples and millions of data points for collaborators leading federated open science initiatives. What's Inside 1,328,973 CDEs organized into 13 themes 768-dim semantic embeddings (SapBERT), IVF4096 FAISS index (sub-second similarity search), Full reproducibility (SHA256 checksums, build scripts), Sources: 9 public standards (OMOP, LOINC, ICD-10, HPO, Mondo, NINDS-CDE, PhenX, CDISC, DICOM, BIDS, DUO) + 9 project collections Get RoP 📥 Data: https://lnkd.in/eyzswQZd (7.8 GB, DOI: 10.57967/hf/8781) 💻 Code & Docs: https://lnkd.in/eJPjmRZS 📋 Questions / Help deploying: https://lnkd.in/e2yhfSWr Community Effort! Primary Authors: Pietro Marini, Rodney Long, Hirotaka Iwaki, Mike Nalls, Dan Vitale Collaborators: Mette Peters, Hampton Leonard, Andy Henrie, Amara Alexander, Elise Marsan, PhD, Yang Fann, Mark Cookson, Cornelis Blauwendraat, Andrew Singleton, Huw Morris, Timothy Hohman, Sarah Biber, PhD, John Crary, Syed Islamuddin Shah, Brittany Dugger, David Gutman, Chris Morris, Patrick Brannelly, Lietsel Jones, Mathew Koretsky, Cole Tindall, Mukta Phatak, Zane Jaunmuktane, Mimi Tambi, Brandon Jernigan, Terri Thompson, Ph.D., PMP, Michael Karlovich, Kurt Kurt Farrell, and many more… CDEs are a community effort. Collaborative Studies: NIH CARD, GP2, NACC, Answer ALS, SEA-AD, ADSP-PHC, ASAP, BDR, BDSA, PART through their connection with the Path-ND Consortium by the 10,000 Brains Project Foundational Concepts: Based on the preprint by Long et al 2024 (https://lnkd.in/e6HtXGwm). Special recognition: We are deeply grateful to the National Library of Medicine (NLM) for their decades of foundational work in biomedical informatics and vocabulary development. We would be honored if any of our harmonization work could contribute back to NLM's mission in any way. #OpenScience #DataHarmonization #FAIR #Bioinformatics #Neuroscience #Genomics #ClinicalResearch

While much of the AI conversation centers on model selection and compute costs, one of the most consequential problems we've observed is far more fundamental — data access. Climate scientists. Genomics researchers. Public health teams. They have the questions. They have the models. But getting to the right data fast enough? That's where the friction lives. This week, AWS open-sourced an MCP server for the Registry of Open Data. Three things this unlocks: Over 1,100 datasets from NASA, NOAA, NIH, and 400+ contributing organizations — now accessible to any AI agent through a single protocol connection. Researchers no longer need to manually navigate storage buckets to find the right dataset. They can ask their AI assistant a natural language question and get a curated answer in seconds. It's open source. Zero cost to connect. Any agent framework, any researcher, anywhere. The organizations solving climate change, curing disease, and protecting communities don't have 18-month data platform budgets. They need to go from research question to dataset to insight in a single afternoon. And that's exactly what MCP makes possible here. We've always believed that AI is not only a great equalizer, but also a great multiplier. This is what that looks like in practice — open data, open protocols, and the right scaffolding so that the science can move at the speed it deserves. The real value of AI won't only show up on earnings calls. It'll show up in the research that changes how we live. #MCP #OpenData #AW