---------------------------------------------------------------------- This is the API documentation for the fcall library. ---------------------------------------------------------------------- ## Core API The three user-facing functions that mirror the R {fcall} package. download_data(year: 'int', month: 'int | str', dest: 'str | Path', files: 'list[str] | None' = None, quiet: 'bool' = False) -> 'None' Download a quarter's FCA Call Report archive and unzip into *dest*. Parameters ---------- year: Four-digit year (e.g. 2025). month: Month name (e.g. ``"September"``) or integer (e.g. ``9``). Valid quarters are 3, 6, 9, 12. dest: Directory to extract files into (created if it does not exist). files: Optional list of file names to extract; ``None`` extracts all. quiet: Suppress progress messages when ``True``. process_data(dir: 'str | Path') -> 'dict[str, Any]' Read a quarter's downloaded .TXT files into tidy Polars DataFrames. Parameters ---------- dir: Path to a directory containing FCA Call Report .TXT files for one quarter (as produced by :func:`download_data`). Returns ------- dict with keys ``"data"`` and ``"metadata"``, each a dict keyed by dataset root name (e.g. ``"RCB"``). compare_metadata(dir1: 'str | Path', dir2: 'str | Path') -> 'dict[str, Any]' Diff the metadata (D_*.TXT) files between two quarter directories. Parameters ---------- dir1, dir2: Paths to directories produced by :func:`download_data`. Returns ------- A dict with two keys: ``"file_differences"`` Sub-dict describing count/name/order differences: - ``"only_in_dir1"`` - filenames present in *dir1* but not *dir2* - ``"only_in_dir2"`` - filenames present in *dir2* but not *dir1* - ``"order_different"`` - ``True`` if the shared files appear in a different order across the two directories ``"content_differences"`` Dict keyed by filename (shared files only). Each value is a list of unified-diff lines for files that differ; files with identical content are omitted. ## Lower-level helpers Exported for power users who need fine-grained control over parsing. Most users should call ``process_data()`` instead. process_metadata_file(file: 'str | Path') -> 'dict[str, Any]' Parse a metadata (D_*.TXT) file into a scenario + vars_info dict. Parameters ---------- file: Path to a ``D_.TXT`` metadata file. Returns ------- ``{"scenario": str, "vars_info": polars.DataFrame}`` The scenario is one of ``"single"``, ``"single_multiple"``, or ``"single_multiple_single"``. process_data_file(file: 'str | Path', metadata: 'dict[str, Any]', codes_dict: 'pl.DataFrame | None' = None) -> 'pl.DataFrame' Read *file* and return a tidy DataFrame using *metadata* and *codes_dict*. Parameters ---------- file: Path to a ``_Q_G.TXT`` data file. metadata: Output of :func:`process_metadata_file`. codes_dict: Code dictionary DataFrame from :func:`get_codes_dict`. Pass ``None`` for datasets with no codes. read_data_file(file: 'str | Path', metadata: 'dict[str, Any]', codes_dict: 'pl.DataFrame | None') -> 'pl.DataFrame' Low-level reader: returns an unnamed DataFrame matching the raw CSV shape. get_codes_dict(data_name: 'str') -> 'dict[str, Any]' Return the codes dictionary for *data_name*, or ``None`` values if none. Parameters ---------- data_name: Root name of the dataset (e.g. ``"RCB"``). Returns ------- ``{"codes_dict": polars.DataFrame | None, "codes_varname": str | None}`` compare_files_content(filename: 'str', dir1: 'str | Path', dir2: 'str | Path') -> 'list[str]' Return unified-diff lines between *filename* in *dir1* and *dir2*. Returns an empty list when the files are identical. ## Data assets Internal datasets shipped with the package. ``fcall.file_metadata`` is a Polars DataFrame (36 rows) mapping file prefixes to descriptions — access it directly. ``get_code_df`` returns a code-to-label DataFrame for datasets that have repeating code groups. get_code_df(registry_key: 'str') -> 'pl.DataFrame' Return a DataFrame[code: Int64, value: Utf8] for *registry_key*. ---------------------------------------------------------------------- This is the User Guide documentation for the package. ---------------------------------------------------------------------- ## Getting Started ### Getting Started `fcall` is a Python package for parsing **Farm Credit Administration (FCA) Call Report** data into tidy [Polars](https://pola.rs) DataFrames. It is a Python re-implementation of the R package [{fcall}](https://github.com/ketchbrookanalytics/fcall) by [Ketchbrook Analytics](https://www.ketchbrookanalytics.com). ## Installation ```bash pip install fcall # or with uv: uv add fcall ``` ## Quick start The package exposes three functions that mirror the R package's public API. ### 1. Download a quarter ```python import fcall fcall.download_data( year = 2025, month = "September", # or month=9 dest = "fcadata/", ) ``` This downloads the September 2025 archive from Ketchbrook's public AWS S3 bucket and unzips it into `fcadata/`. Valid quarters are **March, June, September, and December** (or integers 3, 6, 9, 12). ### 2. Parse the data ```python result = fcall.process_data("fcadata/") # Access a specific dataset rcb_df = result["data"]["RCB"] # Debt Securities inst_df = result["data"]["INST"] # Institution information # Access the schema / metadata for that dataset rcb_meta = result["metadata"]["RCB"] print(rcb_meta["scenario"]) # "single_multiple" print(rcb_meta["vars_info"]) # Polars DataFrame of column definitions ``` ### 3. Compare two quarters ```python fcall.download_data(year=2023, month=9, dest="fcadata_2023/") fcall.download_data(year=2025, month=9, dest="fcadata_2025/") diffs = fcall.compare_metadata( dir1="fcadata_2023/", dir2="fcadata_2025/", ) # Files added or removed print(diffs["file_differences"]) # Line-level content diffs for shared files that changed for filename, diff_lines in diffs["content_differences"].items(): print(filename) print("\n".join(diff_lines)) ``` ## What data is available? FCA publishes Call Report data quarterly. As of March 2026 there are **36 datasets** per quarter (72 files: one metadata `D_*.TXT` and one data file per dataset). See `fcall.file_metadata` for the full list with descriptions: ```python import fcall print(fcall.file_metadata) ``` ::: {.callout-warning} ## 2024 data is broken FCA's 2024 posted files contain a known defect. If you try to process 2024 data, `fcall` will emit a warning and point you to [ketchbrookanalytics/fcall-py/issues/1](https://github.com/ketchbrookanalytics/fcall-py/issues/1) for details and workarounds. ::: ## How-to ### Downloading Data `download_data()` fetches a quarterly FCA Call Report archive from Ketchbrook Analytics' public AWS S3 mirror and extracts it locally. ## Basic usage ```python import fcall fcall.download_data( year=2025, month="September", dest="fcadata/sep2025/", ) ``` ## Month argument `month` accepts either the full month name or its integer equivalent: ```python # Both of these download September 2025 fcall.download_data( year=2025, month="September", dest="sep_by_name/", ) fcall.download_data( year=2025, month=9, dest="sep_by_int/", ) ``` Valid quarters are **3 (March), 6 (June), 9 (September), 12 (December)**. ## Selective extraction Use the `files` argument to extract only the specific file(s) you want to download. ```python fcall.download_data( year=2025, month=9, dest="inst_only/", files=["D_INST.TXT", "INST_Q202509_G20251112.TXT"], ) ``` ## Suppressing output Pass `quiet=True` to suppress the progress messages: ```python fcall.download_data( year=2025, month=9, dest="data/", quiet=True, ) ``` ## URL convention FCA's archive URL format changed in 2015: | Period | URL format | Example | |---|---|---| | 2015 - present | `{year}{MonthName}.zip` | `2020March.zip` | | Before 2015 | `{AbbrevMonth}{year}.zip` | `Sept2011.zip` | `download_data()` handles this automatically. ## Data availability Check to confirm a quarter's data has been published before attempting a download. Typically each quarter's data appears 5-6 weeks after the period ends. ::: {.callout-warning} ## 2024 data is broken FCA's 2024 files contain a known defect. See [ketchbrookanalytics/fcall-py#1](https://github.com/ketchbrookanalytics/fcall-py/issues/1) for details and workarounds while FCA addresses the issue. ::: ## Getting Started ### Data Structure Understanding how FCA structures its Call Report files helps you work effectively with the output of `process_data()`. ## Files in an archive Each quarterly zip contains pairs of files sharing a **root name**: | File pattern | Role | |---|---| | `D_.TXT` | **Metadata** - column names, types, decimal positions, and definitions in a fixed-width file | | `_Q_G.TXT` | **Data** - headerless CSV values | For example, the September 2025 archive contains `D_RCB.TXT` (metadata) and `RCB_Q202509_G20251112.TXT` (data). ::: {.callout-note} Some metadata files in the `RCI2*` family carry an extra `_` suffix (`D_RCI2B_2016.TXT`, for instance) that is **not** present in the data filename. `fcall` handles this automatically. ::: ## The three scenarios `process_metadata_file()` classifies each dataset into one of three **scenarios** based on the structure of the metadata file. ### `"single"` Plain CSV with no repeating column groups. Example: `INST` (institution information). ``` UNINUM NAME STATE ... 12345 Acme CA ... 23456 Beta TX ... ``` ### `"single_multiple"` Each row in the raw file contains data for $N$ *codes* (e.g. investment types). The raw wide format is automatically pivoted by `process_data()` into one row per entity per code. Before pivot (wide): ``` UNINUM INV_CODE__1 AMOUNT__1 INV_CODE__2 AMOUNT__2 ... 12345 10 100.00 15 200.00 ... ``` After pivot (tidy): ``` UNINUM INV_CODE AMOUNT 12345 10 100.00 12345 15 200.00 ``` Datasets in this category (based on March 2026 data): `RCB`, `RCF`, `RCO`, and others. ### `"single_multiple_single"` The raw data file wraps each record across `n_codes + 2` physical lines: leading single columns on line 1, one line per code in the middle, and trailing single columns on the last line. After parsing, the same pivot as `"single_multiple"` is applied. Datasets in this category (based on March 2026 data): `RCR7`. ## Code dictionaries Datasets with repeating code groups use a **code dictionary** to map integer codes to human-readable labels. `fcall` ships 13 such dictionaries (ported from the R package's internal data): ```python import fcall # Look up the code dictionary for a root file name result = fcall.get_codes_dict("RCB") print(result["codes_dict"]) ``` ## Metadata schema `process_metadata_file()` returns a dict with two keys: | Key | Type | Description | |---|---|---| | `"scenario"` | `str` | `"single"`, `"single_multiple"`, or `"single_multiple_single"` | | `"vars_info"` | `polars.DataFrame` | One row per column; see columns below | `vars_info` columns: | Column | Type | Description | |---|---|---| | `ColumnName` | `Utf8` | Variable name (cleaned of `**`) | | `ColumnType` | `Utf8` | `"Numeric"` or `"Alphanum."` | | `DecimalPosition` | `Int64` | Decimal places (0 = integer) | | `Definition` | `Utf8` | Free-text description from FCA | | `MultipleOccurrenceColumn` | `Boolean` | True for repeating columns | | `CodeColumn` | `Boolean` | True for the first repeating column (the code key) | | `ColumnTypeSQL` | `Utf8` | `"text"`, `"integer"`, or `"float"` | ## How-to ### Comparing Quarters `compare_metadata()` diffs the metadata (`D_*.TXT`) files between two quarter directories so you can track schema changes over time. ## Basic usage ```python import fcall # Download two quarters fcall.download_data(year=2014, month=9, dest="q3_2014/") fcall.download_data(year=2025, month=9, dest="q3_2025/") diffs = fcall.compare_metadata("q3_2014/", "q3_2025/") ``` ## Return structure `compare_metadata()` returns a dict with two keys: ### `"file_differences"` Describes changes in which metadata files exist and in what order: ```python fd = diffs["file_differences"] fd["only_in_dir1"] # filenames present in dir1 but not dir2 fd["only_in_dir2"] # filenames present in dir2 but not dir1 fd["order_different"] # True if shared files appear in a different order ``` ### `"content_differences"` A dict keyed by filename (shared files only). Each value is a list of unified-diff lines for files that changed; files with identical content are **omitted**: ```python # Look at all of the content differences across all files for filename, lines in diffs["content_differences"].items(): print(f"--- {filename} ---") fcall.print_diff(lines) # Or look at the content differences for a specific file lines = diffs["content_differences"]["D_RI.TXT"] fcall.print_diff(lines) ``` ## Example: same quarter, no differences Two downloads of the same quarter should show no differences: ```python diffs = fcall.compare_metadata("q3_2025_copy1/", "q3_2025_copy2/") assert diffs["file_differences"]["only_in_dir1"] == [] assert diffs["file_differences"]["only_in_dir2"] == [] assert diffs["content_differences"] == {} ``` ## Comparing a file directly `compare_files_content()` lets you diff a single file between two directories: ```python from fcall import compare_files_content diff = compare_files_content( filename="D_RI.TXT", dir1="q3_2014/", dir2="q3_2025/", ) fcall.print_diff(diff) ```