GSoC 2026: Automated Compilation Verification Across All Concerto Code Generation Languages

Author: Apoorv Gadiya, a contributor to Google Summer of Code 2026

GSoC project: Automated Compilation Verification Across All Concerto Code Generation Languages

Org: Accord Project · Repo: concerto-codegen · Mentors: Diana Lease, Ertugrul Karademir

1. Problem Statement

Concerto is the modeling language of the Accord Project. concerto-codegen generates code and schemas for 16 targets Java, TypeScript, Go, C#, Rust, GraphQL, Protobuf, OpenAPI, Avro, PlantUML, Markdown, and more.

Before this project, CI mostly answered one question:

Did the generator run without crashing?

It did not answer:

Is the generated output valid in its target environment?

That gap is dangerous. A Java visitor can omit an import. A Rust visitor can emit a structure that fails cargo check. A GraphQL visitor can produce SDL that graphql-js rejects. Snapshot tests may still pass, because they only compare text to yesterday’s text.

The GSoC idea (techdocs#483 -CodeGen Test Harness) and my proposal framed three concrete failures in the old setup:

1. No language runtimes in CI -the pipeline lived in Node.js. There was no Java, Rust, Go, or .NET toolchain available to exercise generated code.
2. No compilation / validation step -tests checked that files were created, not that they compiled or validated.
3. No structured verification corpus -there was no shared, versioned set of .cto models driving generate-and-verify across targets in Docker/CI.

Goal: CI should fail immediately when generated output is invalid for any supported target -turning “hope it works” into “verified against tsc / javac / dotnet / cargo / protoc / …”.

2. Solution

The delivered solution matches the three pillars from the proposal, refined with mentor feedback and what actually scaled in the repo:

2.1 Overview

Two complementary layers live inside concerto-codegen (not a separate repo):

All sixteen fromcto targets are covered:

Design rules that kept the harness clear:

1. Generate from a known corpus.
2. Hand artifacts to a real external tool.
3. Fail the job if that tool rejects them.
4. Document intentional skips when a pre-existing visitor bug blocks a case -never weaken the verifier to fake green.

2.2 Approach

Work landed incrementally -one (or a few) targets per PR -so every merge expanded CI coverage without drowning reviewers.

Foundation (#246)
Shared base image, TypeScript + JSON Schema targets, docker-run.js, matrix workflow, local Mocha suites. The base image installs the Concerto CLI and symlinks this checkout’s codegen into the CLI’s module tree so PRs verify branch visitors, not a published npm copy.

Expand the matrix
Compiled targets (C#, Rust, Java, Go) then schema/doc targets (GraphQL, Protobuf, OData, Mermaid, XML Schema, OpenAPI, Avro, PlantUML, Vocabulary, Markdown), in reviewable batches.

Reproducibility
Centralize toolchain pins in verification/docker/versions.json and inject them as Docker build args locally and in CI.

This is same as in the proposal timeline (compiled targets first, schema batch later, matrix + docs last), adapted to how the community actually reviews: many small green PRs beats one giant dump.

2.3 Architecture

Layer 1 -Inputs
  verification/corpus/manifest.json + models/
  test/verification/cases.js (local)

Layer 2 -Images
  verification/docker/base/          # Node + CLI + branch codegen
  verification/docker/<target>/      # toolchain layered on base
  verification/docker/versions.json  # pinned images / npm / jars / crates

Layer 3 -Generate + verify
  run-case.sh - concerto compile --target X
  entrypoint.sh - tsc | javac | go build | protoc | …

Layer 4 -CI gate
  .github/workflows/verify-codegen.yml
  matrix per target, fail-fast: false, failure artifacts uploaded

Container flow

manifest.json
     │
     ▼
for each case - run-case.sh (generate)
     │
     ▼
if artifacts exist - target verifier
     │
     ▼
non-zero exit fails the CI job

Why Docker? Contributors should not need JDK 17, .NET 8, Rust, Go, protoc, and PlantUML JARs installed locally for CI to be trustworthy. Alpine-based images + GHA layer cache keep the matrix practical (the latency risk called out in the proposal).

Why symlink branch codegen? Without replacing nested @accordproject/concerto-codegen under the global CLI, verification would test published codegen and miss PR regressions.

2.4 Coding Examples

Entrypoint shape (every target)

Local builds always pass pinned versions

Dockerfiles take ARGs -no hardcoded tool versions

Compiled targets: multi-stage toolchain copy (Go)

Generation (Node/CLI) and compilation (Go/Java/.NET/Rust) share one container -same pattern for C#, Java, and Rust.

CI matrix

External deps that the proposal flagged (Jackson for Java, serde/chrono for Rust, NuGet for C#) are installed inside those target images not left as “hope the runner has them.”

2.5 Version Pinning

Sixteen Dockerfiles with hard-coded typescript@5.7.2, JAR URLs, and SDK tags drift. Pins now live in one file:

verification/docker/versions.json

docker-versions.js emits --build-arg flags. Local docker-run.js and CI both consume them. Bump once; every image rebuild picks it up.

2.6 How to Run

npm run verify:docker                 # all Docker targets
npm run verify:docker:typescript      # one target
npm run test:verify                   # local Mocha verification

Failures leave artifacts under verification/work-<target>/ (also uploaded by CI). Maintainer docs: verification/VERIFICATION.md.

3. PRs

On accordproject/concerto-codegen by apoorv7g:

Verification harness

Related

Foundation work shipped in v5.1.0.

4. Issues Created

The harness is also a bug radar. Issues filed while standing up compile/schema checks:

Few issues were created after this blog was written, for more information check concerto-codegen-issue-page

Also referenced: techdocs#483 (idea), #257 (DateTime typing – PR #258).

5. Learnings and Acknowledgement

Learnings

Open source is as much about communication and collaboration as it is about writing code. Keeping mentors updated, asking questions, and discussing ideas early made the development process much smoother.

I learned to spend more time understanding the architecture and debugging problems instead of jumping straight into writing code. That shift in mindset has made me a better engineer.

Git is simply a tool, and merge conflicts are a normal part of collaborative development. Don’t be scared of it.

Contributing to a mature open source project taught me the importance of writing maintainable code, accepting feedback, and iterating on solutions through code reviews.

Acknowledgement

This project would not have been possible without the incredible support of the Accord Project community.

A huge thank you to my mentors, Ertugrul Karademir and Diana Lease, for their constant guidance, thoughtful reviews, and encouragement throughout the program. Their feedback not only helped improve my contributions but also helped me grow as a developer.

A special thanks to Matt Roberts for always taking the time to provide detailed feedback and for going above and beyond to help whenever I needed guidance.

I’d also like to thank the reviewers, organization administrators, and everyone in the Accord Project community who took the time to review my pull requests, answer questions, and make new contributors feel welcome. It has been a pleasure contributing to such a collaborative and supportive open source community.

Finally, thank you to Google for continuing to organize Google Summer of Code and making opportunities like this possible. It has been an incredibly rewarding experience, and I look forward to continuing my contributions to the Accord Project even beyond GSoC.