Skip to content

DATA · ML · AI PLATFORM

Systems that hold up when someone asks how they work.

I build data platforms and AI systems that teams can inspect, reproduce and operate: lakehouses, ML pipelines, retrieval systems and the models behind them.

Lucas Rangel at work
Lucas Rangel Soares de Souza
Senior Data & AI Platform Engineer
years building software, data and AI
12+
projects delivered
140+
research and technical articles published
19
companies served
20+

TRY IT

Live demos you can open, test and question.

Each demo covers one area teams hire for, from generative AI to data governance, running on the public Brazilian datasets published on Kaggle. They describe what the records say; they are not legal advice or a supplier recommendation.

  • Live

    Generative AI · RAG

    RAG Chat (opens in a new tab)

    A self-hosted language model with no refusal layer, grounded in knowledge bases: it searches procurement notices, contracts and education spending, writes read-only SQL when the answer is a number, cites every record and keeps the thread of a conversation.

    Open the chat

  • Live

    Vector search · Semantic retrieval

    Procurement explorer

    1.25 million procurement notices embedded with a self-hosted model and indexed in pgvector: compare keyword search with search by meaning, side by side, next to the aggregate dashboard.

    Search notices

  • Live

    BI · Analytics

    Education spending

    An analytics dashboard over municipal education spending: investment per student by year, region and state, built on modelled semantic tables.

    Open the dashboard

  • Live

    Data lake · Catalogue · Governance

    Data map

    The catalogue of a public data lake: 428 tables and 33,891 columns from raw to analytics layers, with source, lineage, join keys and the privacy rules applied, searchable in the browser.

    Browse the map

OPEN SOURCE

The code behind every demo and research project.

Every demo and research project above, grouped by track, with its repository, published data and stack.

DATA PLATFORMS

  • Live Demo

    Brazil public data map

    Eleven Brazilian public sources, from the school census to procurement, released as raw, trusted and analytics layers with contracts, a privacy gate and per-file hashes.

    31 Kaggle datasets, 428 tables and about 4 billion rows of Parquet. Every file is listed in a SHA-256 manifest, and a clean download is verified against it.

    • BigQuery
    • Parquet
    • Python
    • Kaggle API

ML SYSTEMS

  • Live

    Education finance MLOps

    Municipality-level anomaly triage on public education spending, with lineage and a drift gate that refuses to score a shifted batch.

    v0.2.0 scored the 2022 batch with 96 review signals; the drift gate blocked the 2023 batch at a spread ratio of 1.384.

    Outputs are review signals. There are no labels, so no accuracy is claimed.

    • Python
    • scikit-learn
    • MLflow
  • Live

    Procurement ranking

    Transparent retrieval and ranking of historical procurement notices, with every score explained.

    v0.2.0 ranks notices from the pinned PNCP release and verifies its hash before use.

    Offline evaluation uses synthetic profiles, so it measures constraint adherence, not user relevance.

    • Python
    • BM25
    • Evaluation

GENAI AND RETRIEVAL

  • RAG Chat

    A cited research chat over three public bases: text and vector retrieval for notices, read-only SQL for contracts and spending, and gates that run before any model call.

    1,250,335 notice embeddings in pgvector. The 2026-10-01 browser session answered 10 of 10 scripted questions.

    Numeric questions across all bases can still fall back to text retrieval; a question router is in progress.

    • Next.js
    • FastAPI
    • Postgres
    • pgvector
  • Live

    Self-hosted LLM serving

    The 27B Qwen model behind the chat, served from one rented GPU through an OpenAI-compatible API.

    NVFP4 weights on vLLM with speculative decoding: from 4.4 to about 34 tokens per second on the same GPU.

    The abliterated checkpoint is a third-party release; I did not retrain it.

    • vLLM
    • Terraform
    • GitHub Actions

RUNTIME AND DELIVERY

  • Live

    Distributed agent runtime

    Redis-coordinated workers with idempotent requests, Kubernetes manifests, Terraform and recovery tests.

    48 requests at concurrency 6: 61.89 req/s, p95 147.95 ms. Worker recovery proven on a local kind cluster.

    Deterministic model stub on one host; not a capacity claim. Cloud not validated.

    • Redis
    • Kubernetes
    • Helm
    • Terraform

CAREER

Twelve years, from embedded software to AI platforms.

Ten employers across retail, banking, payments, credit and education. Select one to see what I did there.

  1. DEC 2024 – NOW · 1 YR 10 MO

    Drogasil

    Data and AI engineering · Retail

    Drogasil logo

    Data and AI engineering for one of Brazil's largest pharmacy chains: pipelines, models and AI agents delivered in one engineering flow, with governance and data quality built in.

    What I did

    • ELT pipelines with dbt and PySpark, and data prepared for AI applications.
    • Golden ID, deduplication, governance and AI-assisted cataloguing.
    • LLM, RAG and vector-database solutions, with GitLab CI/CD for data, models and agents.

    Results

    • Pipelines, models and AI agents evolving together in one integrated engineering flow.
    • Stronger governance, cataloguing and monitoring of data quality.
    • dbt
    • PySpark
    • Airflow
    • GitLab CI
    • LLM
    • RAG

When each tool entered the work

First year a tool appears in a role above.

  1. 2014

    Java · C++ · Python · MQTT

  2. 2016

    Delphi · Oracle · PL/SQL

  3. 2017

    Spark · Airflow · Azure DevOps · SAP HANA

  4. 2019

    Node.js · React · Docker · AWS · SSIS · SQL Server

  5. 2021

    GCP · BigQuery · Hadoop · Hive

  6. 2022

    Vertex AI · Dataproc

  7. 2023

    Databricks · PySpark · MLflow · Delta Lake

  8. 2024

    dbt · GitLab CI · LLM · RAG

RECENT PROJECT WORK

Client work is described by sector and stack only. Names and results stay with the clients.

  • GROCERY E-COMMERCE

    Product recommendations for a European online grocer, chosen in two stages: first the category, then the brand and pack size.

    AWS Glue · SageMaker Pipelines · DeepFM · DIN · Terraform

  • ENTERTAINMENT

    Concession recommendations for registered and anonymous customers, with an A/B design.

    Azure ML · Microsoft Fabric · Azure DevOps

  • FINANCE

    A multi-agent assistant whose every number comes from a bronze, silver and gold lake, never from the prompt.

    AWS Glue · Athena · SageMaker · LLM agents

  • MINING AND BULK MATERIALS

    Stockpile volume from drone imagery: photogrammetry, ground fitting and point-cloud segmentation in a container job.

    OpenDroneMap · ECS Fargate · Open3D · DBSCAN

  • CREDIT

    A self-hosted data lake and risk engine for receivables financing, scoring financial risk and bad faith on separate axes.

    ClickHouse · MinIO · Airflow · Metabase · FastAPI

  • EDUCATION

    Public-data and product lakes for education companies: raw, trusted and semantic zones, daily orchestration and BI.

    BigQuery · Airflow · Metabase · Power BI

  • LEGACY MODERNISATION

    A specification-driven transpiler that moves COBOL, Delphi and SAS code to Python and Databricks.

    LLM · SDD · Databricks

EDUCATION

Engineering first, then software, then data and AI: each degree matches a step in the career above.

  • PortugueseNative
  • EnglishIntermediate
  1. Bachelor's degree

    Mechatronics Engineering

    Instituto Federal de Goiás (IFG)

    Electronics, control systems, embedded programming and industrial automation: the base of the early work in embedded software and automation.

  2. Technologist degree

    Systems Analysis and Development

    Universidade Norte do Paraná (UNOPAR)

    Software engineering, databases, systems analysis and application development.

  3. Specialisation

    Data Engineering

    Universidade Norte do Paraná (UNOPAR)

    Data pipelines, data modelling, warehouses and big data processing.

  4. Specialisation

    Machine Learning and Artificial Intelligence

    Centro Universitário UNYLEYA

    Machine learning models, their evaluation and applied artificial intelligence.

WRITING

Engineering notes, with the numbers and the limits.

All 19 articles