historical-data-manager

# Historical Data Manager for Construction

## Overview

Manage legacy construction data from archives, old systems, and historical records. Extract, clean, normalize, and migrate data into modern formats for analysis and benchmarking.

## Business Case

Construction companies accumulate decades of project data in various formats:
- Paper records scanned to PDF
- Legacy database exports (Access, dBase, FoxPro)
- Old spreadsheet formats (Lotus 1-2-3, early Excel)
- Proprietary software exports
- Project closeout documentation

This skill helps extract value from historical data for:
- Cost benchmarking and trending
- Productivity analysis over time
- Risk pattern identification
- Estimating improvement

## Technical Implementation

### Historical Data Extractor

```python
from dataclasses import dataclass, field
from typing import List, Dict, Any, Optional
from datetime import datetime
from pathlib import Path
import pandas as pd
import re
import json

@dataclass
class HistoricalRecord:
    project_id: str
    project_name: str
    year: int
    data_type: str  # cost, schedule, labor, material
    original_format: str
    extracted_data: Dict[str, Any]
    quality_score: float
    notes: List[str] = field(default_factory=list)

class HistoricalDataManager:
    """Manage extraction and normalization of historical construction data."""

    def __init__(self, archive_path: str):
        self.archive_path = Path(archive_path)
        self.records: List[HistoricalRecord] = []
        self.normalization_rules = self._load_normalization_rules()

    def scan_archive(self) -> Dict[str, int]:
        """Scan archive and categorize files by type."""
        file_types = {}

        for file_path in self.archive_path.rglob('*'):
            if file_path.is_file():
                ext = file_path.suffix.lower()
                file_types[ext] = file_types.get(ext, 0) + 1

        return file_types

    def extract...