The consultant will implement improvements to the current database handling and management, amongst others but not limited to:
- Operations currently in MS Access should be implemented within an MySQL database with stored procedures
- Implementation of cleaning routines with appropriate logging, using external APIs (e.g. OpenRefine, MediaWiki)
- The contractor will deliver, along with all developments, accompanying documentation as to what has been implemented, including suggestions for further improvements (where applicable) in the form of concrete tasks and estimated effort.
A management application should be introduced, developed in Python with the purpose of:
- Providing an alternative way of implementing computation algorithms that may be too complex when implemented as stored queries or procedure in the database engine
- Introducing a business logic layer that runs outside of the database engine, in order to support the injection in the processing pipeline of features not appropriately supported by a pure-database solution (e.g. the integration of an external HTTP service) and to
- Supporting the automatic execution of the processing pipeline, orchestrating the sequence of processing tasks and assisting the operator in monitoring of the correct process execution, generating alerts for situations that may require manual intervention
