(+61) 0435 158 332
I've been a freelance software engineer for the last 15 years, with a focus on product and user experience—a product engineer. I've worked as a principal, engineering manager, director of a mobile (game) studio, designer and illustrator. This has given me a unique perspective on the process of creating software products.
Ultimately however, no matter the medium, I'm a problem solver.
April 2021 - Now
4th Harmonic is a small defence industry consultancy that has created an in-house product called Maxwell to generate reports with regards to networks between entities in the Australian defence industry. Reporting directly to the CTO, my job was to create Kepler, a crawling and analysis engine that could be fed a table of company URLs, crawl the URLs and generate data that in turn could be fed into Maxwell for data analysis.
After prototyping the initial script to crawl and generate data from URLs, I was tasked with designing a full system to generate and access this data. I designed and authored a 3-layer system, written in Python: an API and database as the main interface, using the FastAPI micro-framework and PostgreSQL; an Azure Functions app that handled the crawling and analysis of data, stored in Azure Storage; and a CLI to access and manipulate the data.
The project was established with source control, functional tests for each layer, CI/deployment using GitHub Actions, and user documentation (data scientists within 4th Harmonic). Despite a seemingly simple goal, the most difficult layer proved to the crawler (Azure Functions), in particular, dividing up the functionality into scaleable and resilient components, whilst having to constrain concurrency due to using a limited scraping proxy service.
Languages & Platforms
Libraries & Frameworks