Network Study Paint By Numbers

The study behind this walkthrough

The worked example is Estimating the risk of non-arteritic ischemic optic neuropathy (NAION) and diabetic retinopathy progression associated with semaglutide exposure, an OHDSI network study led by Cindy X. Cai (Wilmer Eye Institute, Johns Hopkins) with the OHDSI Eye Care and Vision Research Workgroup. It began as a safety question in the news, semaglutide and a rare blinding condition, and became a network study run across administrative claims and electronic health record databases, with results published in JAMA Ophthalmology in 2025.

Every artifact is public:

• The research protocol, including the full cohort definitions and concept sets in its appendix
• The study repository with the executable study package
• The published results (open access, JAMA Ophthalmology 2025)

Design details on this track are reproduced from version 1 of that protocol as posted at ohdsi-studies.github.io/SemaglutideNaion, accessed August 2026. Concept IDs and thresholds may be revised over time, so verify anything you reuse against the live protocol and ATHENA.

The research question

The clinical concern was a single question: does semaglutide cause NAION? The protocol split it into separate questions, each with a stated population, exposure, comparator, and outcome:

1. Among patients with type 2 diabetes mellitus, is the risk for NAION higher among patients who use semaglutide as compared with a non-GLP-1 RA anti-diabetic medication?
2. Among patients with type 2 diabetes mellitus, is the risk for diabetic retinopathy progression higher among patients who use semaglutide as compared with a non-GLP-1 RA anti-diabetic medication?
3. Among patients with type 2 diabetes mellitus, is the risk for NAION higher among patients who use semaglutide as compared with other GLP-1 RA anti-diabetic medications?

Each is a population-level effect estimation question in the standard OHDSI form: a defined population, a defined exposure, a defined comparator, a defined outcome. The study also characterized NAION incidence in each exposure cohort before estimating anything, so a clinical characterization question sits underneath the estimation questions, and a self-controlled case series reuses the same cohorts as a second design.

The same study, run through the use case generator

If you had typed this study into the Analytic Use Case Generator before writing the protocol, it would have produced a use case per question. Here is what goes in each slot, taken from the protocol:

Filling the slots this way shows something the protocol text alone does not. The comparative effectiveness template gets instantiated once per comparator and outcome pair, which is how a short list of protocol questions becomes the much larger set of comparisons the study ran. The self-controlled case series has no template in the generator, because it contrasts exposed and unexposed time within the same people rather than comparing two cohorts.

Feasibility assessment

Before any concept set is built, the use case generator's feasibility questions can be answered from what is already known about the question and the data. Answered from the protocol, they come out like this for the semaglutide study:

That mix sits in the upper feasibility bands: strong on population, comparator, and confounding control, with the outcome phenotype as the one real risk. The study addressed exactly that risk by defining NAION twice and measuring both definitions, which is what the assessment would have told you to do.

The assessment is provisional. Prevalence, coded capture, and follow-up are estimates until the cohorts exist and cohort diagnostics report counts from each participating database. When an assessment comes back weak, the response is not always a new question: a different design, such as a self-controlled case series in place of a cohort comparison, or a different set of data sources can make the same question feasible.

Design and study population

The primary design is an active comparator, new-user cohort design. The population is adults 18 or older with type 2 diabetes mellitus; an indication cohort defines that population and excludes type 1 and secondary diabetes. The target is new users of semaglutide as second-line treatment after metformin. Comparators come in two flavors: within class (dulaglutide, exenatide) and across class (empagliflozin, sitagliptin, glipizide), and the protocol states these were chosen as the most commonly occurring medication within each class in the LEGEND-T2DM study. A self-controlled case series compares NAION incidence during exposed and unexposed time in the same people, which trades confounding-by-indication concerns for within-person assumptions.

The study period runs from December 1, 2017 to December 31, 2023. Analysis uses large-scale propensity scores with 1:1 matching and Cox models for the cohort comparisons, and Poisson regression for the self-controlled case series, executed through the Strategus pipeline calling HADES packages.

Concept sets

A concept set is the bridge between a clinical idea ("type 2 diabetes", "semaglutide") and the rows in an OMOP database. Each entry names a standard concept and two decisions: whether to include its descendants in the vocabulary hierarchy, and whether the entry is an exclusion that subtracts concepts from the set. The protocol appendix publishes every concept set as a table with exactly those columns, which is what an ATLAS export looks like.

The protocol appendix holds the same style of table for every other set the cohorts need: metformin, insulin, each comparator ingredient, and each competing drug class. Every clinical phrase in a cohort definition ("prior metformin", "no prior sulfonylurea", "no type 1 diabetes") is backed by its own named concept set.

Rather than doing that by hand, you can paste the finished set straight in. ATLAS offers two import routes, both on the Import tab of a concept set.

If a pasted JSON ever throws a DataTables warning about an unknown parameter, the JSON is missing a field that ATLAS's table expects. The concept identifier route avoids that class of problem, because ATLAS builds the concept records from the vocabulary rather than trusting the file.

ATHENA and the vocabulary, online and offline

ATHENA is the OHDSI vocabulary browser, and it is where the concept IDs above come from. The working pattern for filling a concept set:

1. Search the clinical term and filter to Standard Concepts = Standard; only standard concepts populate the *_concept_id fields your cohorts query.
2. Pick the right level: for drugs, the RxNorm Ingredient (semaglutide, 793143); for conditions, the SNOMED clinical finding (Diabetes mellitus, 201820). Descendants then do the fan-out to dose forms or subtypes.
3. Open the concept page and review its hierarchy and related concepts to decide what descendants will pull in and what needs excluding.
4. Record the concept ID, the source code, and the vocabulary in your concept set table, with the Excluded and Descendants flags decided deliberately.

ATHENA also distributes the vocabulary tables for download (an account is required, and some vocabularies such as CPT4 need a UMLS API key during the download step). Once loaded into a database, three tables answer almost everything: CONCEPT, CONCEPT_ANCESTOR, and CONCEPT_RELATIONSHIP. The tasks below use the study's real concept IDs, so you can run them against any OMOP vocabulary and compare. Each one is shown twice: the point-and-click path in ATLAS, and the SQL for a downloaded vocabulary. Use whichever fits how you work; they answer the same question.

SELECT concept_id, concept_name, domain_id, vocabulary_id,
       concept_class_id, standard_concept, concept_code
FROM concept
WHERE vocabulary_id = 'RxNorm'
  AND concept_code = '1991302';   -- semaglutide; expect concept_id 793143

-- Type 2 diabetes as defined in the protocol:
-- Diabetes mellitus and descendants, minus T1DM, secondary, and gestational branches
SELECT DISTINCT c.concept_id, c.concept_name, c.vocabulary_id
FROM concept_ancestor ca
JOIN concept c ON c.concept_id = ca.descendant_concept_id
WHERE ca.ancestor_concept_id IN (201820, 443238, 442793, 4016045)
  AND c.invalid_reason IS NULL
  AND c.concept_id NOT IN (
        SELECT descendant_concept_id
        FROM concept_ancestor
        WHERE ancestor_concept_id IN
          (201254, 435216, 40484648, 40484649,
           195771, 761051, 43531009, 4024659)
      )
ORDER BY c.concept_name;

-- Which ICD-10-CM (or other source) codes map to these standard concepts?
SELECT src.vocabulary_id, src.concept_code, src.concept_name,
       std.concept_id AS standard_concept_id, std.concept_name AS standard_name
FROM concept_relationship cr
JOIN concept src ON src.concept_id = cr.concept_id_1
JOIN concept std ON std.concept_id = cr.concept_id_2
WHERE cr.relationship_id = 'Maps to'
  AND cr.invalid_reason IS NULL
  AND std.concept_id IN (793143)   -- swap in any resolved standard IDs
ORDER BY src.vocabulary_id, src.concept_code;

Cohort logic and time-at-risk

Here is the semaglutide exposure cohort from the protocol, laid out as the anatomy every ATLAS cohort definition shares. Read it top to bottom as a machine would.

The same structure repeats for each comparator with the drug names swapped. The logic is identical; only the concept sets change.

The outcome side gets the same rigor. NAION has two definitions, a specific one (two ischemic optic neuropathy diagnoses on separate days within 90 days, with the outcome date pulled earlier if preceded within 60 days by a visual field defect, optic disc disorder, optic neuritis, or optic disc edema) and a sensitive one (a single diagnosis with the same adjustments); both exclude giant cell arteritis and traumatic optic neuropathy. Rather than assuming these definitions work, the study measured their performance with PheValuator.

Windows and time-at-risk

Every window in this study is anchored to the cohort's start or end date, which is why they are designed here with the cohort logic rather than afterward. They are stored in two different places, though. The washout and the lookbacks are part of the cohort definition in ATLAS. The time-at-risk is not part of a cohort definition; it is entered in the analysis specification as the risk window start and end settings of the HADES modules, with each one anchored to cohort start or cohort end. Because the on-treatment time-at-risk ends at cohort end, the exit rule above is what defines it.

• The washout window is the 365 days of continuous observation required before entry, set on the entry event. It gives every subject enough visible history to establish "new user" and to build covariates.
• The lookback windows inside the inclusion rules (metformin up to 90 days before entry; class exclusions anytime before entry; the insulin rule's 30-day spans) each encode a specific clinical requirement.
• The time-at-risk is when outcomes count. The primary analysis is on-treatment: from cohort entry until a gap in exposure of more than 30 days, the end of continuous observation, or the end of the study period, whichever comes first. An intent-to-treat analysis is run as well, and a sensitivity analysis splits the study period into calendar windows (December 2017 to January 2020, February 2020 to June 2021, July 2021 to December 2023) because semaglutide uptake changed so much across those years.

Cohort diagnostics

Once the cohorts exist, the next check is whether the definitions behave as intended in real data. The protocol runs CohortDiagnostics on every target, comparator, indication, and outcome cohort before any estimation. It reports cohort counts, inclusion rule attrition, incidence over time, the index event breakdown, visit context, cohort overlap, and orphan concepts, which are concepts found in the data that appear related to a concept set but are not included in it. It gives evidence for review rather than a pass or fail verdict.

For the outcome, the study went a step further: PheValuator estimated sensitivity, specificity, and predictive values for both NAION definitions, replacing "we assume the codes are right" with measured performance.

Where the fixes go

Problems found here send you back to the definitions, and the fix can sit in either one. Orphan concepts and unexpected source codes point to the concept sets. Heavy attrition on one inclusion rule, or an index event breakdown dominated by something unexpected, points to the cohort logic.

Diagnostics at every site

A definition that behaves well in one database can behave differently in another, so diagnostics are reviewed for each participating data source, not only the coordinating site's. At your own site, the CohortDiagnostics R package can run before the study package is complete. At other sites, it runs from the package's diagnostics-only specification, which is why the protocol and package are drafted alongside the definitions and finalized afterward.

Finalizing the protocol

Drafting the protocol starts at the design stage and grows as each stage is settled. This stage finalizes it, and in OHDSI the finished protocol is posted publicly before any results exist. The SemaglutideNaion protocol is posted as a versioned HTML document, and its structure is the community's working template, so copy it:

• Front matter: abbreviations, responsible parties with institutional affiliations, and a disclosure that the study runs within OHDSI.
• Abstract: background, purpose, design, subjects, methods, and expected conclusions, written in the same shape a journal will later ask for.
• Amendments and milestones: empty sections held open, so later changes are recorded as amendments rather than quiet edits.
• Rationale and background: the trial and observational evidence that motivated the question, with citations.
• Study objectives: the questions restated as numbered objectives.
• Research methods: design, population, exposures, outcomes, covariates, and negative controls, each pointing at the cohort definitions.
• Data analysis plan: which HADES packages run, in what order, with the time-at-risk definition written out.
• Study diagnostics: the pass or fail thresholds applied after execution, stated before any result is seen.
• Strengths and limitations, human subjects protection, dissemination plan: including that each site seeks its own institutional review board (IRB) determination.
• Appendix: every cohort definition in full, printed as entry events, inclusion rules, exits, and concept set tables.

That appendix is what makes the protocol reproducible. A reader can rebuild every cohort from the document alone, without access to the authors' ATLAS instance.

Finalizing and testing the study package

The protocol describes the study; the study package executes it. Like the protocol, the package starts early, since a preliminary version already runs cohort diagnostics at other sites, and this stage finalizes and tests it. For this study that package is an R project in a public repository, generated from the OHDSI EmptyStudyRepository template, so its shape matches other network studies:

How a site tests before running

1. Set up R, RTools, RStudio, and Java following the HADES R Setup instructions.
2. Confirm that R can reach the OMOP CDM through DatabaseConnector, before touching the study code.
3. Download the package, open the project, and restore the environment with renv::restore(). The README warns this step is slow and asks you to reopen the project afterward.
4. Run the diagnostics specification first, by swapping which JSON the script loads. This generates the cohorts and their diagnostics without running the estimation.
5. Set minCellCount to your site's own censoring requirement before any results leave.
6. Run the full specification only once diagnostics look right.

Site readiness

A site can run the package only once the following are in place. Most are conditions of joining any network study rather than steps specific to this one:

• An OMOP CDM conversion of the site's data, on a CDM version the package supports.
• A data quality assessment, such as the OHDSI Data Quality Dashboard and the Achilles characterization reports, so the coordinator can judge whether the database suits the question.
• Approvals: the site's own IRB determination and any data governance sign-off its institution requires. The SemaglutideNaion protocol states that each site seeks its own IRB determination.
• A working R environment with the HADES packages, and a confirmed connection to the CDM through DatabaseConnector.
• The package and upload credentials, with the key file and user name issued by the study coordinator.

Approvals often take the longest, so sites start them while the package is still being finalized.

Running it across the network

This stage is what makes it a network study. Every site runs the same specification against its own OMOP CDM database, and patient-level data never leave the site. Each site writes results into a per-database folder as CSV files, reviews them locally, and uploads only approved aggregate results to the coordinator, with small counts censored below the minimum cell count at the source.

Study diagnostics and their thresholds

For the estimation analyses, the protocol set a second layer of diagnostics with numeric thresholds, written in advance and applied to the executed results while blinded to the estimates. These are computed from the study's own output, so they can only be checked after execution. A database contributes to the meta-analysis only if it passes.

The self-controlled case series has its own gates: a pre-exposure check and a time-trend check, each requiring p above 0.05, plus the same EASE bound. Residual bias is estimated from negative controls, exposure-outcome pairs with no expected causal effect, adapted from LEGEND-T2DM through a data-driven algorithm plus expert review and used to calibrate the estimates.

Synthesis

Once results arrive, the coordinator applies these gates and runs the evidence synthesis (evidenceSynthesis.R in this package) across the databases that pass. Meta-analysis fits this study because it estimates effects. A characterization study usually reports its results database by database instead, so synthesis is a design choice rather than a required step.

Interpreting and publishing the results

Results are published through the OHDSI results explorer, discussed with the workgroup and on the community calls, presented at conferences, and written up as a manuscript. Much of the manuscript's methods section can be adapted from the finalized protocol, which already describes the design, cohorts, and analysis in the shape a journal asks for, and the manuscript can cite the protocol's posted version.

This study reached the last of those stages: the findings appeared in JAMA Ophthalmology, and the repository badge moved to Results Available. A companion study on diabetic retinopathy followed from the same package.