A Minimal Basis for Chemical Structure Comprehension and Editing

The ability to understand, reason about, and modify chemical structure is central to medicinal chemistry and structure-based drug discovery. Accordingly, chemical structure comprehension is a critical component for an agentic LLM system for small-molecule design. In this post we will explore chemical structure comprehension by developing a tree-based data structure and tools for agents to work with 2D and 3D chemical structures. Then we will test the framework against baseline LLM text editing and generalist LLM with bash tool calling.

Basis

Typical chemical structure data formats were not designed for LLM comprehension. For example, while multiple SMILES strings can represent a given 2D molecule, their sensitivity to atom, bond, and ring placement presents challenges for logical editing by agents. Moreover, medicinal chemistry applications require reasoning over structures hypothesized to bind with a conserved orientation. Optimizing a ligand series to better fit a hydrophobic surface requires identification of which bond vector is oriented in the correct direction, and which lipophilic functional groups might better fill the pocket. Swapping a central scaffold for another, sometimes called core hopping, would force a text-only agent to recapitulate every substituent’s relative placement around the new ring.

Considering these scenarios presents an interesting question: What is the minimal basis for text-based small-molecule structure comprehension in medicinal chemistry? Here we propose the following:

• Stable references - A persistent address per atom and functional group so instructions and feedback correspond across ligand hypotheses.
• Conventional decomposition - The fragments provided as LLM context must correspond to typical conversational medicinal chemistry.
• Paired 2D and 3D information - A representation that keeps the graph and pose in sync so that 2D edits have well-defined 3D consequences.
• Editing operations - A set of tools that enable an agent to arbitrarily edit chemical structures.

Chemical Fragment Tree Data Structure

Chemists tend to refer to ring systems and functional groups when referencing chemical substructures. Examples: “let’s consider swapping the piperazine for a morpholine”, “the naphthalene is not very drug-like, what if we changed it to a quinazoline”, or “what if we fluorinated ortho to the methyl”. Therefore, the natural data structure for stable references, conventional decomposition, and paired 2D / 3D information is a chemical fragment tree. If we fragment molecules by systematically breaking acyclic single bonds, maintaining their connectivity and 3D coordinates, we arrive at a break-order invariant tree. The broken bonds are maintained as “ports” for reconstruction of the original molecule and edited analogs. Every group, port and atom is labeled with a stable ID, and coordinates are stored along with the fragments. Rings and functional groups stay whole and referenceable.

Figure 1. ABL1 crystal ligand fragment tree before and after swapping a group.

This data structure was implemented as the core of chemistree, a Python project written specifically to explore the concepts in this post.

Minimal Tool Set

The fragment tree data structure provides the key abstraction, but we still require a set of tools that allow LLMs to discover and act upon the information. In this section we summarize a minimal set of tools that was built to enable general 2D and 3D structure-manipulation tasks. Importantly, many of these tools were built experimentally using test-driven development by augmenting capabilities until a baseline model (Haiku) was able to succeed in basic tasks. For example, during experimentation it became clear that LLMs were not very good at inferring the IUPAC ring-positioning commonly used by chemists to refer to substitution patterns. Accordingly, this domain knowledge is given as a set of named heteroaromatic rings with IUPAC numbering in tool calls and returned context.

The following tools are provided to agents via a model context protocol (MCP) server.

Inspection:

• describe - List the nodes in the fragment tree and ports connecting them. Also provides basic molecular properties for medicinal chemistry reasoning (MW, cLogP, TPSA, HBD, HBA, rotatable bonds, aromatic rings, Fsp3, formal charge, structure alerts).
• describe_group - Show one group’s atoms with their position ids, its rings, and each atom’s bonded neighborhood.
• matches - Search the current molecule for a substructure (a heterocycle name or SMILES/SMARTS) to confirm an edit produced the intended outcome.

Editing:

• swap - Replace the whole group at a node with a new one, named as a common group name or a SMILES.
• grow - Add a group in place of a chosen hydrogen.
• mutate - Change a single heavy atom’s element (e.g. a ring carbon to nitrogen).
• remove - Delete a leaf group, capping its former attachment point with hydrogen.
• undo - Revert the most recent edit.

3D / Binding Site:

• distance - Report each group’s distance to a named receptor residue, closest first, with per-atom detail.
• contacts - Map the binding site: the closest group and atom for every residue near the ligand.
• residues_near - List the receptor residues near a chosen group (or one of its atoms).
• clashes - List the steric clashes in the current pose.
• minimize - Rotate a group about its attachment bond to its lowest-energy rotamer.

Writing:

• smiles - Return the canonical SMILES of the current molecule.
• write_pose - Save the current 3D pose to an SDF file.

For testing, the standalone MCP server also exposes bind which loads a molecule SMILES or SDF plus an optional receptor into a session.

When chemistree is given a 3D ligand and protein receptor, the describe tool also provides information about the predicted binding affinity and ligand strain (internal energy) to provide feedback during structure optimization.

Benchmarking

The key question for benchmarking to answer is does the proposed tree data structure and minimal tool set enable LLMs to understand and edit structure better than a generalist agent using the same model?

The following LLM configurations were tested with Haiku (smallest Anthropic model) using a harness built around Claude Code:

• naked - baseline model with no tools that generates answers by directly reasoning over the SMILES (2D only).
• generalist - agentic competitor provided with Bash, rdkit, and smina for docking.
• chemistree - MCP tools only.

Tasks

A subset of 11 of the established DUD-Z docking benchmark set was selected as the test set for this post. The crystal ligands were used for 2D comprehension and editing tasks. The posed ligands and PDB structure were used for 3D comprehension and optimization tasks. All prompts were generated by iterative refinement with Claude Opus 4.8 and manual spot checking. Care was taken such that all answers were computed independently via rdkit, no tool-specific information was included in the task prompts, and only questions with unique correct answers were included. The ABL1 case was used for scoping question formats and developing the benchmarking harness. Accordingly, it was checked more thoroughly and then used as an example to generate the remainder of the tests (more) automatically. This step took some refinement and for a published study we would want to take more care to check every question and answer. We would also make a 100% human chemist generated test set for comparison and control.

The tasks can be broken down into the following categories:

• 2D Editing: Molecular structure editing. Success is defined as identifying the unambiguous SMILES (canonicalized internally) associated with a set of prompts.
• 3D Decoration: Strip the ligand to its Murcko scaffold but maintain its pose. Then re-optimize its predicted binding. Success is defined as improving predicted binding over the bare scaffold measured by Smina Vinardo score redocking.
• 3D Comprehension: Deterministic Q&A over the fully posed crystal ligand, matched to a computed truth. This task tests reasoning over a small molecule posed in a binding site.

Figure 2. The crystal ligand of each of the 11 benchmark targets.

Table 1. DUD-Z benchmark example task prompts. ABL1 is the hand-curated scoping set; the other targets’ prompts were generated automatically and spot-checked.

Testing Setup

For fair comparison, tests should give the different LLM configurations identical task prompts. The system prompts must deviate to account for the different tools available but should otherwise be comparable (e.g., all have the same general guidelines and persona). For transparency we save the system and task prompt for every evaluation along with the results and to prevent Claude from cheating we stage a working directory holding only what it is allowed to see (e.g., so it cannot find the ground truth in the repository). Each prompt was asked in triplicate to give a preliminary assessment of stochasticity. For each question and replicate we measure the correctness of the response and cost / tokens.

As a control for 3D decoration tasks, we also include runs without strategy guidance to help separate representation / tools from coaching. In these unguided runs the shared medicinal-chemistry playbook and the tool-usage guidance (everything in Figure 3 below each arm’s access-mechanics line) is removed, leaving only the access mechanics.

Figure 3. System prompts. See table 1 for task prompts.

Statistics

The main statistics we will track are accuracy / completion fractions which are reported over all cases (including replicates). In addition, we carry out a basic analysis of significance via case-level pairing. One nuance is that LLM inference is stochastic, so we have replicate runs. The three replicates are not independent samples, so they are collapsed to per prompt solved count for testing (don’t want to inflate significance). A two-sided sign test is used where the p-value is the binomial probability over discordant cases, meaning it uses only cases where the arms disagree and discards ties. No multiple comparisons correction is applied.

Outcomes

The overall benchmark outcome is measured across all tasks based on the above success definitions. The naked (no tool) model was only tested on 2D tasks. Statistics are summarized in Table 2. Overall, chemistree displayed the highest aptitude across all tasks with an 88% success rate compared to 68% for naked (over its 2D cases only, p = 0.012) and 70% for generalist (p = 0.0002). However, it also displayed the highest cost (dominated by 3D decoration; when omitted it is the lowest).

Table 2. Overall benchmark outcomes (N=3 per prompt). Input tokens are the mean uncached input only. Footprint is the full context processed (uncached, cache-create, cache-read).

In the 2D benchmark subset, success was defined based on unambiguous canonical SMILES for the requested edits, testing both comprehension and manipulation of molecules represented as SMILES strings. The results are summarized in Table 3. In this task set chemistree achieved an 89% success rate compared to 68% for naked (same as above, p = 0.012) and 64% for generalist (p = 0.0018). An analysis across task categories (Table 4) reveals the improvement was concentrated in growing and core-hopping whereas simple substitutions were handled comparably well by the generalist (25 / 30 vs 29 / 30). Interestingly, the generalist’s core-hopping success rate was less than the no tool model suggesting access to cheminformatics capabilities does not improve core comprehension and analogous substitution. In this task set chemistree emits the fewest output tokens but processed the most cached context. However, due to the lower cost (~1/10) for cached context reads the average cost for chemistree was marginally lower than both other models.

Table 3. 2D benchmark outcomes (N=3 per prompt). Input tokens are the mean uncached input only. Footprint is the full context processed (uncached, cache-create, cache-read).

Table 4. 2D benchmark success by category.

In the 3D decoration benchmark, subset success was defined based on improving the predicted binding affinity over the unsubstituted scaffold (built by removing substituents from the crystal ligands) and measured by a Smina Vinardo redock. This task attempts to measure the model’s ability to design structures with improved properties. In this section both guided (including a medicinal chemistry strategy in the prompt) and unguided (letting agents formulate their own strategy) were tested as a control. The results are summarized in Table 5. In this task guided chemistree had the highest success rate at 73% and returned a dockable molecule on nearly every case (32 / 33). However, with only 11 paired targets we cannot reject the null hypothesis of equivalent performance (p = 0.125 vs guided generalist), and among molecules that both arms docked the mean binding gain was comparable. This task’s cost dominated the overall cost average for chemistree.

Table 5. 3D decoration benchmark outcomes (N=3 per prompt). Non-obvious columns: (completion) fraction of runs that returned a dockable molecule, (recovery) Mean ECFP Tanimoto similarity between the optimized structure and the crystal ligand.

In the 3D comprehension probe, success was defined by matching the exact answer on queries regarding a posed crystal ligand. This task evaluates both comprehension of ligand structure and its relation to residues in a binding site. The results are summarized in Table 6. In this task both chemistree and the generalist performed well with 95% and 92% success rates respectively (the difference is not significant, p = 0.45). However, chemistree was able to arrive at an answer at lower cost.

Table 6. 3D comprehension probes (N=3 per prompt). Input tokens are the mean uncached input only. Footprint is the full context processed (uncached, cache-create, cache-read).

Molecular Design Web Application

We have established that chemistree shows strong performance at chemical structure comprehension and editing tasks. In this section we highlight a minimal collaborative interface via a light-weight local web application to visually showcase its capabilities and reasoning. The same MCP tool set drives the local web application which features 2D structures and the corresponding 3D poses. The views update automatically as you converse with the agent in the chat panel (with optional speech-to-text). Users can make specific natural-language requests like “swap the oxazole for a phenyl” or general tasks like “optimize the predicted affinity”. As changes are made a film roll shows the chain of edits which can be toggled into the visualizers by clicking.

Figure 4. Web application demonstration with ABL1 with Haiku. The session starts with a prompt for a description of the loaded ligand. Then the agent is prompted to remove substituents, swap the core to a quinazoline, and add back new substituents to improve predicted binding.

Conclusions

This study was designed for a small budget, both in tokens and time (N=3, 11 targets draw from an established benchmark, underpowered statistics, largely auto-generated test cases – albeit spot checked), so please take all conclusions as anecdotal. A hand curated test set, more targets, higher N, and more model types are the obvious next steps toward a powered study. Still there are some interesting preliminary insights that can be draw from the experiments discussed in this post.

• chemistree beats the generalist overall (p = 0.0002) and on 2D editing (p = 0.0018). On 3D comprehension and 3D decoration the differences are smaller and not significant at this sample size. The direction favors chemistree, but we would need more experiments to know for sure.
• The generic bash / rdkit agent was statistically indistinguishable from the no tools baseline on the 2D editing tasks. However, chemistree clearly outperformed no tools. This shows that having the right representation, coupled to tools that act on it, can improve performance. General cheminformatics software alone is not enough.
• The task category where the chemistree had the most impact was core hopping. Navigating relative substituent placement around a swapped core in SMILES is obviously challenging.
• chemistree improves binding on substantially more targets than the generalist (24 / 33 vs 15 / 33 guided) and returns a dockable molecule nearly every time (32 / 33 guided, 33 / 33 unguided). Among molecules that both arms dock, however, the per-molecule binding gain is comparable.
• On 3D comprehension both chemistree and the generalist perform well (95% and 92%, difference not significant). This may be a sign that the chosen benchmark queries are well-connected to the standard representation (“which residue has an atom closest to this functional group” is easy to compute from 3D coordinates if you correctly parse the functional group).
• chemistree did not increase the cost over a no tools model in 2D editing. However, it executed more cycles and was more expensive than the generalist in 3D design.
• I tried to fully leverage loops and automation to build chemistree, the benchmark, and run experiments. However, perhaps not surprisingly, when left to its own devices Claude engineered some helpful hints into the system prompt and tool docstrings which might have given chemistree an edge. Accordingly, some steering and review was required to arrive at a fairer contest.

You can check out the project and benchmarks here. I recommend playing around with the local web application, it’s a hoot.