πŸŽ“

SQL Server DB Python Builder

Project completed in 7 times less weeks than originally estimated (86% time reduction)

πŸ“…

Before: Weeks
(Manual tedious work)

β†’
⚑

Now: Seconds!
(Error-free automation)

πŸ’‘

Focus here is functionality, not coding practices.

These projects were built over a year ago with the sole purpose of solving real business problems efficiently. They demonstrate my ability to identify repetitive tasks and automate them effectively. The goal was not to create a perfect project, but a fast solution. Do not evaluate my coding standards or practices based on them.

Project Context

🏒
Setting

Data-specialized consultancy in Spain

πŸŽ“
Client

One of Spain's most prestigious universities

πŸ“Š
The Challenge

Build database structure with 350+ tables in allocated timeframe

πŸ’‘
My Approach

Automated the entire process with Python instead of manual creation

⚑
Development

Few days to build
the automation script

πŸš€
Task Impact

7x faster completion than
allocated project timeline
86% time reduction

πŸ”„
Future Value

Future changes done in seconds
compared to weeks of manual work

πŸ—„οΈ About the Database Structure

⚠️ The Challenge

Build a new database structure with 350+ tables that needed to be created manually, one by one, across 4 different layers:

1. SSIS Packages

Extract data from Oracle to SQL Server

2. STG Layer

SQL Server First layer: staging tables

3. ODS Layer

SQL Server Second layer: operational data store

4. Stored Procedures

Data transformations STG β†’ ODS

Database Migration Architecture
graph TD subgraph "Oracle Source Systems" A[ORIGIN_1 Database<br/>πŸ“Š Tables & Data] B[ORIGIN_2 Database<br/>πŸ“Š Tables & Data] end subgraph "Python Automation Engine" C[πŸ“‹ Data Dictionary<br/>350+ Tables Metadata] D[🐍 Python Builder<br/>Auto-generation Script] end subgraph "Generated Components" E[πŸ“¦ SSIS Packages<br/>XML Code Generation] F[πŸ—„οΈ SQL Server STG<br/>Staging Layer] G[πŸ—„οΈ SQL Server ODS<br/>Operational Data Store] H[βš™οΈ Stored Procedures<br/>ETL Transformations] end A --> E B --> E C --> D D --> E D --> F D --> G D --> H E --> F F --> H H --> G classDef oracle fill:#ff9999,stroke:#ff3333,stroke-width:2px classDef python fill:#4CAF50,stroke:#2E7D32,stroke-width:2px classDef generated fill:#2196F3,stroke:#1565C0,stroke-width:2px class A,B oracle class C,D python class E,F,G,H generated
⏰ Manual Time Requirements

350+ tables Γ— 20 columns average = 7,000+ individual elements to create manually across 4 layers:

Per Table Estimates: (Work to do for each one of the +350 tables)
  • β€’ SSIS Package: 45-60 minutes
  • β€’ STG Table: 15-20 minutes
  • β€’ ODS Table: 15-20 minutes
  • β€’ Stored Procedure: 30-45 minutes

Total per table: ~2 hours

⚠️ Additional Manual Process Challenges:
  • β€’ Column name variations: Many tables have different naming conventions requiring manual checking
  • β€’ Data type mismatches: Oracle to SQL Server type conversions need individual verification
  • β€’ Human error risk: Repetitive manual tasks are highly prone to mistakes and inconsistencies
  • β€’ Version control issues: Manual changes across 350+ tables create maintenance nightmares
πŸš€ The Solution Impact

My Python automation eliminated this extremely tedious and time consuming manual process. Instead of creating each table manually across all layers, the script generates everything automatically: SSIS packages with complex XML structures, SQL table definitions, and stored procedures with transformationsβ€”all in seconds rather than weeks, with zero human error risk.

πŸ”§ Technical Challenges & Achievements

1. Complex SSIS Package Generation
  • β€’ Programmatically created SSIS packages for 350+ tables
  • β€’ Each package containing hundreds of lines of XML code
  • β€’ Automated weeks of manual effort into seconds of execution
  • β€’ Ensured precise XML structure adherence for SSIS functionality
2. Multi-layered Database Structure
  • β€’ Managed table creation across multiple database layers (STG, ODS)
  • β€’ Maintained data integrity and consistency across layers
  • β€’ Implemented complex transformations between layers
3. Cross-Database Compatibility
  • β€’ Translated Oracle-specific features to SQL Server equivalents
  • β€’ Handled different data types and constraints across platforms
  • β€’ Ensured data integrity across different database systems

πŸ“‚ GitHub Repository: View Complete Source Code β†’

πŸ“© Ready for Your Own Transformation?

By leveraging technology to solve real-world problems, I help clients optimize their operations and focus on what truly matters.

πŸ’¬ Interested in a similar transformation?

Let's chat about how I can tailor a solution for your specific needs!

⚠️ Confidentiality: For confidentiality reasons, I can't provide the university name or exact time details, but both can be fact-checked during the recruitment process.