Claude Code transcript

Based on analyzing the last 50 commits and the codebase structure, here are the key challenges and potential improvements:

Why This Codebase Is Difficult

1. Massive Monorepo Scale

• 37 top-level services/domains (ledger, census, teller, banking, onboard, etc.)
• 12,574 Go files with 4,209 test files
• 1,003 generated mock files and 232 proto files
• Single go.mod means changes can ripple across services

2. Inconsistent Commit Message Standards

• 8+ different ticket systems: LP, APPROV, CSENG, SPND, PAY, DX, INC, RC
• 16% of commits (8/50) are NOJIRA/no-jira - no traceability
• Inconsistent formatting: [LP-1267], LP-1267 -, (FEAT)[CSENG-200]
• Bot commits (Update job's newest image version) add noise

3. Cross-Service Coupling

• Many commits touch 3-5 services simultaneously (census+teller+ledger is common)
• Changes in internal/commons/ affect all services
• Ledger alone touched 42 files in the last 50 commits

4. High Bug Fix Rate

• 8 commits (~16%) are explicit fixes: panics, race conditions, orphaned data, broken tests
• Pattern: "Fix integration test setup script by properly killing orphaned SSH connections"
• Suggests fragile test infrastructure

5. Heavy Workflow/Temporal Migration

• 13/50 commits (26%) involve workflows/Temporal/jobs
• Active migration from cron jobs to Temporal workflows adds cognitive load

6. Deep Directory Nesting

Typical path: teller/internal/workflow/create_payments_for_payment_plans/impl.go - Layers: service/internal/layer/feature/impl/file.go - Hard to navigate; easy to put code in wrong place

How to Make It Easier

Quick Wins

1. Standardize commit messages: Enforce [PROJ-123] verb: description format via pre-commit hook
2. Require ticket links: Block NOJIRA commits on main (or at least require justification)
3. Filter bot commits: Tag automated commits so they can be filtered from history

Medium-Term Improvements

1. Service ownership docs: Each service's AGENTS.md should declare owners and primary ticket prefix
2. Dependency visualization: Generate a service dependency graph to understand coupling
3. Integration test reliability: Several commits fix test infrastructure - invest in stability

Larger Initiatives

1. Consider service boundaries: Services like census, ledger, and teller have tight coupling (frequently changed together). Either:
2. Accept they're one logical unit
3. Or invest in cleaner API boundaries
4. Reduce mock proliferation: 1,003 mock files suggests over-mocking. Consider table-driven tests or integration tests where appropriate
5. Flatten directory structure: internal/domains/feature/impl/ could often just be internal/feature/

The core issue: this is a large, fast-moving monorepo with ~22 contributors, multiple teams (8 ticket systems), and services that are tightly coupled in practice despite being logically separated.

Now I have a clear picture. Here's the analysis:

The 20 Most Frequently Changed Files

Root Causes of Merge Conflicts

1. Monolithic Proto Files

census.proto has 98 RPCs and 224 messages in a single file. Every new endpoint modifies this file.

Fix: Split by domain:

census/api/grpc/v1/
  product_account.proto    # Product account RPCs
  consumer_profile.proto   # Consumer profile RPCs
  debt_sale.proto          # Debt sale RPCs
  membership.proto         # Membership RPCs

2. Centralized Dependency Wiring

server_init.go (2,214 lines) must be edited every time you add a new client, DAO, or handler. Every InitXxx function is a conflict magnet.

Fix: Use dependency injection registration pattern:

// ledger/internal/domains/creditmanager/module.go
func init() {
    server.RegisterDomain(func(s *Server) error {
        s.CreditManager = creditmanager.New(s.SQLClient, s.Census)
        return nil
    })
}

Domains register themselves; no central file to edit.

3. Feature Flag Interface Explosion

teller/internal/feature/contract.go (43 lines, 32 methods) grows with every feature flag. Each addition: - Modifies the interface - Modifies the impl - Regenerates the mock (7 changes in 50 commits)

Fix: Use a generic feature flag pattern:

type Store interface {
    GetBool(ctx context.Context, key string, defaultVal bool) bool
    GetInt(ctx context.Context, key string, defaultVal int64) int64
    GetJSON(ctx context.Context, key string, out interface{}) error
}

// Usage - no interface change needed:
if store.GetBool(ctx, "enable_payment_plan_job_processing", false) { ... }

4. God Object Domain Managers

product_account/impl/impl.go (1,210 lines) handles too many responsibilities. Recent changes added statement deletion logic, ledger client dependencies, etc.

Fix: Decompose into focused services:

census/internal/domains/product_account/
  create.go       # CreateProductAccount, CreateLoanProductAccount
  delete.go       # DeleteProductAccount, HardDeleteProductAccount  
  update.go       # UpdateProductAccount
  query.go        # GetProductAccount, ListProductAccounts

5. Handler Registration Churn

census/internal/svc/service.go requires adding a new handler field for every new handler type.

Fix: Use a handler registry:

type Server struct {
    handlers map[string]Handler
}

func (s *Server) Register(name string, h Handler) {
    s.handlers[name] = h
}

Quick Wins (Low Effort, High Impact)

1. Split census.proto into 4-5 domain-specific files. gRPC supports imports.

2. Generic feature flags - Replace the 32-method interface with 3 generic methods. Eliminates mock regeneration.

3. Auto-register domains - Use Go's init() pattern to have domains register themselves instead of editing server_init.go.

4. Extract generated files from repo - Move z_mocks_*.go and z_swagger*.json to a generated directory with a .gitattributes merge strategy: **/z_*.go merge=ours **/z_*.json merge=ours

These four changes would eliminate ~60% of the merge conflict surface area based on the commit patterns.

Now I have the full picture. Here are concrete examples:

1. Auto-Register Domains

The Problem Today

Every time someone adds a new domain manager to ledger, they must:

1. Add a field to server.go (line ~140-170)
2. Add an initializeXxxManager() method to server_init.go
3. Add a call to that method in InitDomains() (line ~987-1075)

This means 3 locations in 2 files that everyone editing domains must touch. When two people add domains in parallel → merge conflict.

Current Pattern (server_init.go:1026-1032)

func InitDomains(ctx context.Context, s *Server, config *config.Config) error {
    // ... 20+ other initializations ...

    if err := s.initializeDelinquencyManager(); err != nil {
        return err
    }

    if err := s.initializeCreditLineOverrideManager(); err != nil {
        return err
    }

    // ... 10+ more initializations ...
}

Every new domain adds 3 lines here. Two PRs adding domains = guaranteed conflict.

Proposed Pattern: Self-Registering Domains

Step 1: Create a registry (new file: ledger/internal/server/registry.go)

package server

import "context"

// DomainInitializer is a function that initializes a domain and attaches it to the server.
type DomainInitializer struct {
    Name string
    Init func(ctx context.Context, s *Server) error
}

// domainRegistry holds all registered domain initializers.
// Domains register themselves via init() functions.
var domainRegistry []DomainInitializer

// RegisterDomain adds a domain initializer to the registry.
// Called from init() functions in each domain package.
func RegisterDomain(name string, init func(ctx context.Context, s *Server) error) {
    domainRegistry = append(domainRegistry, DomainInitializer{Name: name, Init: init})
}

// InitAllDomains runs all registered domain initializers.
func InitAllDomains(ctx context.Context, s *Server) error {
    for _, d := range domainRegistry {
        if err := d.Init(ctx, s); err != nil {
            return fmt.Errorf("failed to initialize domain %s: %w", d.Name, err)
        }
    }
    return nil
}

Step 2: Each domain registers itself (e.g., ledger/internal/domains/delinquencymanager/register.go)

package delinquencymanager

import (
    "context"
    "github.com/go-backend/ledger/internal/server"
)

func init() {
    server.RegisterDomain("delinquency", initDelinquencyManager)
}

func initDelinquencyManager(ctx context.Context, s *server.Server) error {
    manager, err := NewManager(
        WithLogger(s.Logger()),
        WithStats(s.Stats()),
        WithCreditLineDAO(s.SQLDao().CreditLineDAO),
        WithTermsDAO(s.SQLDao().TermsDAO),
        WithFeatureStore(s.FeatureStore()),
    )
    if err != nil {
        return err
    }
    s.SetDelinquencyManager(manager)
    return nil
}

Step 3: Replace InitDomains with the registry call

// server_init.go - InitDomains becomes:
func InitDomains(ctx context.Context, s *Server, config *config.Config) error {
    return InitAllDomains(ctx, s)
}

Step 4: Import all domain packages in one place (ledger/internal/server/domains.go)

package server

// Import all domain packages to trigger their init() registration.
// Add new domains here - this is the ONLY file that needs editing.
import (
    _ "github.com/go-backend/ledger/internal/domains/collectionmanager"
    _ "github.com/go-backend/ledger/internal/domains/creditlineoverridemanager"
    _ "github.com/go-backend/ledger/internal/domains/creditmanager"
    _ "github.com/go-backend/ledger/internal/domains/creditprofilemanager"
    _ "github.com/go-backend/ledger/internal/domains/delinquencymanager"
    _ "github.com/go-backend/ledger/internal/domains/feeprofilemanager"
    _ "github.com/go-backend/ledger/internal/domains/loanmanager"
    // ... etc
)

Why This Is Better

The key insight: import lists merge cleanly because git can auto-merge additions to different lines. Sequential if err := ...; return err blocks cannot.

2. Extract Generated Files from Git Merge Conflicts

The Problem Today

Generated files like z_mocks_*.go and z_swagger*.json are: - Changed in 15+ commits out of the last 50 - Regenerated whenever their source changes - Cause merge conflicts that are meaningless (just regenerate!)

Example: Two PRs both add a method to census.proto. Both regenerate z_mocks_census_grpc_client.go. Merge conflict. But the resolution is always: regenerate from merged proto.

Solution: .gitattributes Merge Strategy

Create .gitattributes in repo root:

# Generated mock files - always accept "ours" on merge, then regenerate
**/z_mock*.go merge=ours
**/z_mocks_*.go merge=ours

# Generated swagger files - same strategy
api/swagger/z_swagger*.json merge=ours
api/swagger/z_swagger*.yaml merge=ours

# Generated proto files
**/*.pb.go merge=ours

How merge=ours works:

When git encounters a conflict in these files during merge/rebase: 1. Instead of showing conflict markers, it keeps "our" version 2. The merge succeeds without manual intervention 3. CI then regenerates the files from source (proto, interfaces)

Required CI step (add to PR workflow):

# .github/workflows/pr-checks.yml
- name: Regenerate and verify generated files
  run: |
    # Regenerate mocks
    go generate ./...

    # Regenerate swagger
    ./scripts/generate-and-validate-swagger.sh

    # Fail if there are uncommitted changes (generated files out of sync)
    git diff --exit-code || (echo "Generated files out of sync. Run 'go generate ./...' and commit." && exit 1)

Alternative: Don't Commit Generated Files At All

A more aggressive approach: add generated files to .gitignore and generate them in CI/locally.

.gitignore additions:

# Generated mocks - regenerate with 'go generate ./...'
**/z_mock*.go
**/z_mocks_*.go

# Generated swagger - regenerate with './scripts/generate-and-validate-swagger.sh'  
api/swagger/z_swagger*.json
api/swagger/z_swagger*.yaml

Makefile target for local dev:

.PHONY: generate
generate:
    go generate ./...
    ./scripts/generate-and-validate-swagger.sh

.PHONY: setup
setup: generate
    # Other setup steps...

Why This Is Better

Recommendation: Start with merge=ours + CI validation. It's less disruptive than removing files from git entirely, but eliminates the merge conflict problem.

Concrete Before/After

Before (typical merge conflict in z_mocks_census_grpc_client.go):

<<<<<<< HEAD
func (_m *MockCensusClient) ListCreditLinesByDebtSale(ctx context.Context, ...) {
=======
func (_m *MockCensusClient) BulkUploadDebtSaleCreditLine(ctx context.Context, ...) {
>>>>>>> feature-branch

Developer must: understand both changes, manually merge, hope they got it right.

After (with merge=ours):

# Merge succeeds automatically
# CI runs go generate
# CI commits regenerated file (or fails if source is wrong)

No manual intervention. The generated file is always correct because it's regenerated from the merged source.

Migration Path

Week 1: Add .gitattributes for generated files

• Zero risk, immediate benefit
• Merge conflicts in z_* files disappear

Week 2-3: Pilot domain registry in ledger

• Create registry.go
• Migrate 2-3 domains to self-registration
• Keep old InitDomains working for non-migrated domains

Week 4+: Roll out to other services

• Apply same pattern to census, onboard, teller
• Each service can migrate incrementally