Data for AI

I have experience using Python, Requests, Scrapy to crawl sites and extract publicly available information.
I have built projects with an entire ETL extract-transact-load pipeline and stored in Postgres, MySql and T-SQL for clients. All projects are built and pushed to private git repos and tested with CI/CD workflows.
For more performant processing of data I now look to use Rust and I have built projects with SurrealDB and used an emedded KV store as well.