Aditya-fix: Restore Cost Breakdown by Category API for Financials donut chart

[Aditya-gam]

Description

Restores and completes the Cost Breakdown by Category backend so the donut chart (Dashboard → Reports → Total Construction Summary → Financials) can render. The costs API existed (PR #1300), but the router was never registered in routes.js, so endpoints were unreachable. This PR mounts the router, refactors around a materialized view and cost aggregation service, fixes route ordering, and adds unit tests.

• Router: Register costs router at /api/costs in src/startup/routes.js.
• Model: Cost schema gains costDate, projectName, projectType, indexes; unique key (projectId, category, costDate).
• Controller: Uses BuildingProject for validation; getCostBreakdown supports projectId, startDate, endDate, categoryDetail=true; getCostsByProject at GET /project/:projectId (paginated, optional category); admin-only CRUD + POST /refresh; cache with clearByPrefix on write/refresh.
• Service: costAggregationService.js — runCostAggregation(projectIds?) builds costs from BuildingProject (labor), BuildingMaterial, BuildingTool (approved purchases); bulk upsert into costs.
• Tests: costsController.spec.js, costAggregationService.spec.js, costsRouter.test.js.

Related PRs

• Current Frontend PR: PR #
• Previous PR pair: PR #3659 + PR #1300

Main changes explained

Created/Updated Files

• src/startup/routes.js

  • Added: app.use('/api/costs', costsRouter); so the costs router is mounted and all cost endpoints are reachable at /api/costs/*. Previously, the router was required but never registered.

• src/models/costs.js

  • Schema: projectId as ObjectId ref buildingProject; added costDate (required, indexed), projectName, projectType (enum: commercial/residential/private), calculatedAt, lastUpdated, source (aggregation/manual/correction). Category enum: Total Cost of Labor, Total Cost of Materials, Total Cost of Equipment.
  • Indexes: Unique compound { projectId, category, costDate } for aggregation upserts; { projectId, costDate }, { costDate } for breakdown and date-range queries.
  • Exports: Cost, COST_CATEGORIES, DEFAULT_HOURLY_RATE (25).

• src/controllers/costsController.js

  • Dependencies: Uses BuildingProject (bmdashboard) for project validation and names; costAggregationService.runCostAggregation for refresh; nodeCache with clearByPrefix('cost_breakdown:') and clearByPrefix('costs_project:') for invalidation.
  • getCostBreakdown: Validates projectId (ObjectId, must exist in BuildingProject), startDate/endDate (valid dates, start ≤ end). Cache key: cost_breakdown:${projectId||'all'}:${startDate}:${endDate}:${categoryDetail}. Returns { project, totalCost, breakdown }; with categoryDetail=true, each breakdown item includes projectBreakdown: [{ projectId, projectName, amount, percentage }].
  • addCostEntry / updateCostEntry / deleteCostEntry: Admin-only (req.body.requestor.role in ['Administrator','Owner']). Validate category, amount ≥ 0, projectId/costId; invalidate cache on success.
  • getCostsByProject: Validates projectId (params), optional page, limit (capped at 100), category. Returns { costs, pagination } with totalCosts, totalPages, currentPage, hasNextPage, hasPreviousPage. Cached per project/category/page/limit.
  • refreshCosts: Admin-only; body projectIds optional array; calls runCostAggregation(projectIds || null), then invalidates cost caches; returns { message, updated, errors }.

• src/routes/costsRouter.js

  • Route order: Static routes before parameterized to avoid /breakdown and /refresh being matched as IDs: GET /breakdown → getCostBreakdown; POST /refresh → refreshCosts; POST / → addCostEntry; PUT /:costId, DELETE /:costId → update/delete; GET /project/:projectId → getCostsByProject. Ensures GET /api/costs/project/:projectId is used for project cost list (not GET /api/costs/:projectId).

• src/services/bmdashboard/costAggregationService.js

  • runCostAggregation(projectIds?): Loads building projects (all or by projectIds); for each project: computes labor from members[].hours * DEFAULT_HOURLY_RATE, materials from BuildingMaterial approved purchaseRecord (quantity × unitPrice by date), equipment from BuildingTool approved purchaseRecord; builds bulk upsert ops by (projectId, category, costDate); runs Cost.bulkWrite (upsert). Returns { updated, errors }; per-project failures are caught and pushed to errors, and aggregation continues.
  • Helpers: calculateProjectLaborCost(project, hourlyRate), aggregateApprovedPurchasesByDate(inventoryDocs), buildUpsertOps(projectInfo, category, costsByDate).
  • triggerProjectAggregation(projectId): Debounced (30s) single-project aggregation for future event-driven sync (e.g. after material/tool approval or time-log update).

• src/controllers/costsController.spec.js

  • Unit tests for all controller methods with mocked Cost, BuildingProject, cache, logger, costAggregationService. Covers: getCostBreakdown (validation errors, cache hit, no projectId/projectId, categoryDetail, aggregation error); addCostEntry (admin check, validation, project not found, success, save error); updateCostEntry / deleteCostEntry (admin, invalid id, not found, success, errors); getCostsByProject (validation, cache, pagination, category filter, error); refreshCosts (admin, projectIds validation, success with/without projectIds, runCostAggregation error).

• src/services/bmdashboard/costAggregationService.spec.js

  • Unit tests for runCostAggregation (no projects, with projectIds, labor/materials/tools calculation, bulkWrite, per-project and top-level errors) and for helpers (calculateProjectLaborCost, aggregateApprovedPurchasesByDate, buildUpsertOps); optional tests for triggerProjectAggregation debounce behavior.

• src/routes/costsRouter.test.js

  • Route wiring tests: GET /breakdown, POST /refresh, POST /, PUT /:costId, DELETE /:costId, GET /project/:projectId call the correct controller methods (e.g. via mocked controller or integration-style request).

Key implementation details

• Materialized view: The costs collection is the single source for chart and project-cost reads. It is populated by runCostAggregation, which derives data from buildingProjects (member hours → labor), buildingMaterials (approved purchase records → materials), and buildingTools (approved purchase records → equipment). One document per (projectId, category, costDate); upsert key prevents duplicates.
• BuildingProject vs Project: Controller and validation use BuildingProject (bmdashboard), so projectId refers to building projects; project name for breakdown label comes from BuildingProject.name.
• Cache keys: Breakdown: cost_breakdown:${projectId||'all'}:${startDate}:${endDate}:${categoryDetail}. Project costs: costs_project:${projectId}:${category||'all'}:${page}:${limit}. Invalidation uses cache.clearByPrefix('cost_breakdown:') and cache.clearByPrefix('costs_project:') on add/update/delete/refresh.
• categoryDetail: When categoryDetail=true, aggregation groups by category and projectId, then attaches projectBreakdown with per-project amount and percentage per category for frontend slice-click drill-down.
• Admin: Admin-only endpoints expect req.body.requestor.role in ['Administrator', 'Owner']; no separate middleware file change in this PR.

---

How to test

1. Check out the branch: Aditya-feature/cost-breakdown-donut-chart-wip
2. Reinstall dependencies and clean cache using rm -rf node_modules package-lock.json && npm cache clean --force
3. Run npm install to install dependencies, then start the backend locally (npm run dev)
4. Use Admin/Owner login.
5. Test cost endpoints (Postman or curl):
   • Breakdown – all projects:
     GET http://localhost:4500/api/costs/breakdown
   • Breakdown – one project:
     GET http://localhost:4500/api/costs/breakdown?projectId=654946c8bc5772e8caf7e963
   • Breakdown – with detail (for slice click):
     GET http://localhost:4500/api/costs/breakdown?categoryDetail=true
   • Breakdown – date range:
     GET http://localhost:4500/api/costs/breakdown?startDate=2024-01-01&endDate=2025-12-31
   • Breakdown – invalid projectId:
     GET http://localhost:4500/api/costs/breakdown?projectId=invalid → 400 "Invalid project ID format"
     GET http://localhost:4500/api/costs/breakdown?projectId=507f1f77bcf86cd799439099 (valid ObjectId but non-existent project) → 400 "Project not found"
   • Project costs (paginated):
     GET http://localhost:4500/api/costs/project/654946c8bc5772e8caf7e963
   • Project costs – pagination:
     GET http://localhost:4500/api/costs/project/654946c8bc5772e8caf7e963?page=1&limit=10
6. Verify:
   • After a successful refresh, call GET /api/costs/breakdown again; response should reflect updated data (or empty breakdown if no costs).
   • Run tests: npx jest src/controllers/costsController.spec.js src/services/bmdashboard/costAggregationService.spec.js src/routes/costsRouter.test.js --coverage --collectCoverageFrom='src/controllers/costsController.js' --collectCoverageFrom='src/services/bmdashboard/costAggregationService.js' --collectCoverageFrom='src/routes/costsRouter.js' --collectCoverageFrom='src/models/costs.js'

---

Screenshots or videos of changes

• Test Coverage:

• Test Video:

TestVideo.mov

---

Note

• **Response shape of GET /breakdown and GET /project/:projectId is backward-compatible with the contract from PR Zhifan - Create a Donut Chart for Cost Breakdown by category backend #1300; the path for project costs is explicitly GET /project/:projectId (not GET /:projectId), so any client using the old path must use the new one.
• Cost schema adds fields and indexes; existing Cost documents with the old shape may need a one-time aggregation run (POST /api/costs/refresh) or migration if present. New deployments can run a refresh to backfill.
• Breakdown and project-cost reads are cached; aggregation can be heavy for all projects. Refresh is admin-only and can be run asynchronously or scheduled for the future.
• projectId/costId validated as ObjectId; projectId must exist in BuildingProject; startDate/endDate validated and start ≤ end; category must be in COST_CATEGORIES; amount ≥ 0.
• Backward compatibility: Compatible with existing auth and nodeCache usage; frontend (PR #3659) can point to GET /api/costs/breakdown and GET /api/costs/project/:projectId once this PR is merged.