# ExtractCodeElements

> **CocoIndex v1.** This page documents CocoIndex **v1** — a ground-up redesign from v0. When writing code, ignore any v0 flow-builder DSL or deprecated decorators.
>
> Source: https://cocoindex.io/docs/ops/code_ast/ · Docs index: https://cocoindex.io/docs/llms.txt · Agent skill: https://cocoindex.io/docs/skill.md
>
> v0→v1 quick map — if you reach for these v0 symbols, stop and use the v1 form: `@cocoindex.flow_def`/`FlowBuilder` → `coco.App` + a `@coco.fn` main function; `add_collector()`/`collect()`/`export()` → declare target states (`declare_row`, `declare_file`); `cocoindex.sources/functions/targets.*` → connector APIs (`localfs.walk_dir`, `coco.ops.*`, `postgres.declare_table_target`). Full mapping + API reference: https://cocoindex.io/docs/skill.md.

The `cocoindex.ops.code_ast` module wraps a TreeSitter-based extractor that
parses source code and returns two structured lists per file:

- **Declarations** — every class, function, method, struct, etc., with its
  namespace, fully qualified entity name, normalized
  [kind](#declaration-kinds), raw AST node kind, and source position. Only
  module-, namespace-, and type-level declarations are indexed; declarations
  local to a function body (nested functions, local classes) are skipped as
  noise. References inside those bodies are still captured.
- **References** — every call site, object instantiation, type parameter, and
  base type that points at another named entity.

```python
from cocoindex.ops.code_ast import ExtractCodeElements
```

**Note — CocoIndex Plus only**
`ExtractCodeElements` ships with `cocoindex-plus`. No extra install — the
TreeSitter grammars are bundled into the wheel:

```sh
pip install cocoindex-plus
```

## Usage shape

`ExtractCodeElements` is constructed once and reused — the extractor caches
compiled grammars and exclusion regexes per language. Call `.extract(...)`
inside a `@coco.fn` to walk one file's source:

```python
import cocoindex as coco
from cocoindex.ops.code_ast import ExtractCodeElements
from cocoindex.ops.text import detect_code_language
from cocoindex.resources.file import FileLike

_extractor = ExtractCodeElements()

@coco.fn
async def index_file(file: FileLike, target: coco.TableTarget) -> None:
    language = detect_code_language(filename=file.file_path.path.name)
    if language is None:
        return
    elements = _extractor.extract(await file.read_text(), language=language)
    for d in elements.declarations:
        target.declare_row(...)  # one row per declaration
    for r in elements.references:
        target.declare_row(...)  # one row per reference
```

The end-to-end example in
[`cocoindex-plus-examples/code_elements_indexing`](https://github.com/cocoindex-io/cocoindex-plus-examples/tree/main/code_elements_indexing)
walks a GitHub repository, runs the extractor over every supported source file,
and writes one Postgres row per declaration and reference.

## `ExtractCodeElements`

```python
ExtractCodeElements(
    *,
    languages: dict[str, CodeElementsLanguageConfig] | None = None,
)
```

| Parameter   | Default | Description                                                                              |
| ----------- | ------- | ---------------------------------------------------------------------------------------- |
| `languages` | `None`  | Per-language overrides (see below). If `None`, the built-in defaults are used as-is.     |

When `languages` is provided, **only the languages named in the map are
enabled**, and the supplied config replaces the built-in node-kind maps and
exclusion patterns. The TreeSitter grammar and language hooks always come
from the built-ins, so the map keys must be names with built-in support (any
of the names in the [Built-in language support](#built-in-language-support)
table; matched case-insensitively).

### `extract()`

```python
extract(
    code: str,
    *,
    language: str,
    base_namespace: str | None = None,
) -> CodeElements
```

| Parameter        | Default | Description                                                                                  |
| ---------------- | ------- | -------------------------------------------------------------------------------------------- |
| `code`           | —       | Source code to parse.                                                                        |
| `language`       | —       | Language name. Case-insensitive. Unknown languages return an empty `CodeElements`.           |
| `base_namespace` | `None`  | Seed namespace component (e.g. the file's Python module path). Ignored by languages like C#. |

## Built-in language support

Defaults ship for the languages below. The `language=` keys match the names
returned by [`detect_code_language`](/ops/text#detect_code_language), so a
language detected from a filename can be passed straight to `.extract(...)`.

| Language       | `language=`    | Captures                                                                                                          |
| -------------- | -------------- | ---------------------------------------------------------------------------------------------------------------- |
| **Python**     | `"python"`     | Classes & functions; calls, typed parameters, base classes. Built-in types (`int`, `str`, `list`, …) excluded.   |
| **C#**         | `"csharp"`     | Classes, structs, interfaces, enums, records, methods, constructors; invocations, object creation, parameter & base / generic-argument types. Block + file-scoped namespaces. |
| **JavaScript** | `"javascript"` | Classes, functions, methods; calls, `new` expressions, `extends` base class. Also covers `.jsx`.                 |
| **TypeScript** | `"typescript"` | JS kinds + interfaces, enums, type aliases; parameter types, `extends` / `implements` bases. `namespace` scopes. Built-in types excluded. |
| **TSX**        | `"tsx"`        | Same as TypeScript, using the `.tsx` grammar.                                                                     |
| **Go**         | `"go"`         | Types (structs/interfaces), functions, methods; calls, struct literals, parameter types. Predeclared types (`int`, `string`, …) excluded. |
| **Rust**       | `"rust"`       | Structs, enums, unions, traits, functions, `impl` blocks; calls, parameter types. `mod` scopes (`::`-joined). Primitive types excluded. |
| **Java**       | `"java"`       | Classes, interfaces, enums, records, methods, constructors; invocations, object creation, parameter, `extends` / `implements` types. `package` namespace. Primitive types excluded. |
| **Kotlin**     | `"kotlin"`     | Classes, interfaces, objects, functions; calls, supertypes. `package` namespace.                                 |
| **Swift**      | `"swift"`      | Classes/structs/enums/extensions, protocols, functions; calls, conformances/supertypes.                          |
| **C**          | `"c"`          | Structs, unions, enums, function definitions; calls. Built-in types excluded.                                    |
| **C++**        | `"cpp"`        | C kinds + classes; calls, parameter types, base classes. `namespace` scopes (`::`-joined). Also covers `.cc`, `.h`, `.hpp`. |
| **Objective-C** | `"objc"`      | `@interface` / `@implementation` / `@protocol`, methods (selectors reassembled, e.g. `initWithFoo:bar:`), C structs/enums/functions; message sends (`[obj method]`), C calls, superclass & adopted protocols. Covers `.m`. |
| **Objective-C++** | `"objcpp"`  | Objective-C kinds **plus** C++ classes, `namespace` scopes, and `qualified_identifier` references — the full Objective-C++ grammar. Covers `.mm`. |

**Note — Single-file approximation**
The extractor works from one file's syntax tree with no cross-file semantic
analysis, so references are a best-effort approximation: `referenced_full_path`
is the syntactic path as written (e.g. `helper.Process`), not a resolved
fully-qualified symbol. Use `exclude_reference_patterns` to trim noisy paths.

**Note — Objective-C++**
`.mm` (Objective-C++) is handled by a dedicated, vendored Objective-C++ grammar
that ships in the `cocoindex-plus` wheel. The lighter Objective-C grammar
(`"objc"`) is used for `.m`.

To restrict or extend the defaults, pass a `languages={...}` map — for
example, to index Python only and drop the built-in `class_base` and
`typed_parameter` reference kinds:

```python
from cocoindex.ops.code_ast import (
    CodeElementsDeclarationConfig,
    CodeElementsLanguageConfig,
    CodeElementsReferenceConfig,
    ExtractCodeElements,
)

extractor = ExtractCodeElements(
    languages={
        "python": CodeElementsLanguageConfig(
            declaration_node_kinds={
                "class_definition": CodeElementsDeclarationConfig(
                    name_field="name", body_field="body", kind="class"
                ),
                "function_definition": CodeElementsDeclarationConfig(
                    name_field="name", body_field="body", kind="function"
                ),
            },
            reference_node_kinds={
                "call": CodeElementsReferenceConfig(path_expr_field="function"),
            },
            exclude_reference_patterns=[r"print|len|range"],
        ),
    },
)
```

## Config types

### `CodeElementsLanguageConfig`

```python
CodeElementsLanguageConfig(
    declaration_node_kinds: dict[str, CodeElementsDeclarationConfig] = {},
    reference_node_kinds: dict[str, CodeElementsReferenceConfig] = {},
    type_list_node_kinds: dict[str, CodeElementsTypeListConfig] = {},
    namespace_node_kinds: dict[str, CodeElementsNamespaceConfig] = {},
    exclude_reference_patterns: list[str] = [],
)
```

The four `*_node_kinds` maps each key an AST node type — as reported by the
TreeSitter grammar for that language — to its extraction rule.
`exclude_reference_patterns` is a list of regex patterns; any reference whose
`referenced_full_path` fully matches one of them is dropped (patterns are
`|`-joined and anchored as `^(?:p1|p2|...)$`).

### `CodeElementsDeclarationConfig`

```python
CodeElementsDeclarationConfig(
    name_field: str,
    body_field: str | None = None,
    kind: str = "others",
)
```

| Field        | Description                                                                                                        |
| ------------ | ------------------------------------------------------------------------------------------------------------------ |
| `name_field` | Named-child field whose text becomes the declaration's `base_name`.                                                |
| `body_field` | Named-child field that holds the body. Used by the language hooks to compute `has_body` (e.g. `... → False`).      |
| `kind`       | Normalized [declaration kind](#declaration-kinds) to emit for this node type. Defaults to `"others"`.              |

### `CodeElementsReferenceConfig`

```python
CodeElementsReferenceConfig(path_expr_field: str)
```

| Field             | Description                                                                                  |
| ----------------- | -------------------------------------------------------------------------------------------- |
| `path_expr_field` | Named-child field that holds the path expression (e.g. `"function"` for a Python `call`).    |

### `CodeElementsTypeListConfig`

```python
CodeElementsTypeListConfig()
```

A marker config — node kinds in `type_list_node_kinds` emit one reference per
named child (used for nodes like C#'s `base_list` and `type_argument_list`).

### `CodeElementsNamespaceConfig`

```python
CodeElementsNamespaceConfig(name_field: str)
```

| Field        | Description                                                                            |
| ------------ | -------------------------------------------------------------------------------------- |
| `name_field` | Named-child field that holds the namespace name (e.g. `"name"` for C# `namespace X`).  |

## Output shape

### `CodeElements`

```python
CodeElements(declarations: list[Declaration], references: list[Reference])
```

### `Declaration`

```python
Declaration(
    namespace: str,
    entity_name: str,
    parent_entity_name: str | None,
    base_name: str,
    kind: str,
    ast_node_kind: str,
    has_body: bool,
    start: CodePosition,
    end: CodePosition,
)
```

| Field                | Description                                                                                  |
| -------------------- | -------------------------------------------------------------------------------------------- |
| `namespace`          | Namespace at the declaration site (e.g. `"MyApp.Services"`, or the `base_namespace` seed).   |
| `entity_name`        | Fully qualified name within its namespace (e.g. `"OrderService.PlaceOrder"`).                |
| `parent_entity_name` | Enclosing declaration's `entity_name`, or `None` if top-level.                               |
| `base_name`          | Simple (unqualified) name of the declaration.                                                |
| `kind`               | Normalized, **cross-language** classification (e.g. `"class"`, `"method"`, `"type_alias"`) — see [Declaration kinds](#declaration-kinds). |
| `ast_node_kind`      | Raw TreeSitter node type (e.g. `"class_definition"`, `"method_declaration"`); differs per grammar. |
| `has_body`           | Whether the declaration has a meaningful body (excludes `...` / empty interface methods).    |
| `start` / `end`      | Source positions — see [`CodePosition`](#codeposition) below.                                |

#### Declaration kinds

`kind` is a classification that is **stable across languages**, unlike
`ast_node_kind` (the raw TreeSitter node type, which differs per grammar) — so
you can select, say, every method without knowing each language's node names.
The vocabulary is:

`class`, `interface`, `struct`, `union`, `enum`, `trait`, `type_alias`,
`function`, `method`, `constructor`, `property`, `constant`, `variable`,
`field`, `extension`, and `others` (catch-all).

Most constructs map to the obvious kind (a class → `class`, a struct →
`struct`, an enum → `enum`). Two things are not a one-to-one rename of the
syntax:

- **`method` vs `function`** is *contextual*, and holds for every language: a
  function declared directly inside a type body is a `method`; a free function —
  or one nested inside another function — stays `function`. The same applies to
  `field` vs `variable`.
- A handful of constructs normalize in a **language-specific** way, where one
  language's keyword is folded into a kind named after another's. These are the
  non-obvious cases worth knowing:

  | Source construct          | Language(s)        | `kind`      |
  | ------------------------- | ------------------ | ----------- |
  | `record`, `record class`  | C#, Java           | `class`     |
  | `record struct`           | C#                 | `struct`    |
  | `protocol`                | Swift, Objective-C | `interface` |
  | `trait`                   | Rust               | `trait`     |
  | `object` (singleton)      | Kotlin             | `class`     |
  | `impl` block, `extension`, category | Rust, Swift, Objective-C | `extension` |

  Whether a record is class-like or struct-like is genuinely a per-language call
  (C# splits on the `struct` keyword; Java records are always reference types;
  a Pascal/Delphi `record` would be struct-like), so it is spelled out here
  rather than stated as a universal rule. This is intentionally just the
  non-obvious cases — not an exhaustive construct-by-language matrix.

`field` and `property` cover the data members of a type, split by language. A
member value is a `property` in the languages that have first-class properties —
C# (`property_declaration`), Kotlin (`val`/`var`), Swift (stored & computed
properties, including protocol requirements), TypeScript (interface members), and
Objective-C (`@property`) — and a `field` everywhere else: C/C++/Objective-C
struct fields, C# fields, Go struct fields, Java fields, Rust struct fields, and
JS/TS class fields. A single declaration that binds several names (`int x, y;`,
Go `X, Y int`) yields one `field` per name. To enumerate all of a type's data,
union `field` and `property`.

`variable` and `constant` cover module-, namespace-, and type-level value
bindings (locals inside function bodies are dropped by the scope filter).
`constant` is reserved for compile-time constants: Go `const`, Rust `const`, C#
`const`, Java `static final`, and Kotlin `const val`. `variable` covers the rest:
Go package `var`, Rust `static`, and JS/TS module-level `const`/`let`/`var` —
JavaScript/TypeScript `const` is binding immutability, not a compile-time
constant, so it is `variable`, not `constant`.

A declaration's enclosing namespace or module is captured in its `namespace`
field rather than emitted as a separate declaration. New kinds are added together
with the extraction that produces them, so the list above is exactly what the
built-in configs emit.

### `Reference`

```python
Reference(
    namespace: str,
    parent_entity_name: str | None,
    referenced_base_name: str,
    referenced_full_path: str,
    ast_node_kind: str,
    start: CodePosition,
    end: CodePosition,
)
```

| Field                  | Description                                                                          |
| ---------------------- | ------------------------------------------------------------------------------------ |
| `namespace`            | Namespace at the call site.                                                          |
| `parent_entity_name`   | Enclosing declaration's `entity_name`, or `None` at module / file scope.             |
| `referenced_base_name` | Simple (unqualified) name of the referenced entity.                                  |
| `referenced_full_path` | Full dotted path (e.g. `"helper.Process"`, `"a.b.do_thing"`).                        |
| `ast_node_kind`        | TreeSitter node type (e.g. `"call"`, `"invocation_expression"`, `"type_argument_list"`). |
| `start` / `end`        | Source positions — see [`CodePosition`](#codeposition) below.                        |

### `CodePosition`

```python
CodePosition(char_offset: int, line: int, column: int)
```

| Field         | Description                              |
| ------------- | ---------------------------------------- |
| `char_offset` | 0-based character offset from file start.|
| `line`        | 1-based line number.                     |
| `column`      | 1-based column number.                   |