Russian  English  All projects

Microservice for parsing and caching (php)



It is required to implement a microservice for data caching preferably in php.

Access to the microservice should be made by http requests with the identifier of the requested data in the get parameter.

At each request, in accordance with the specified parameters, the microservice checks in its storage / cache the presence and relevance of the requested data.
The cache can be implemented based on files or mysql.

If the data is available and the cache lifetime has not passed, then the data is given as an answer as is.
If there is no data, the new data is prepared and stored in the cache and then the data is issued as a response.
If no data is available and the preparation of new data fails, the microservice returns the error {data:[],message:"Error"}
If the data is available, but the cache lifetime has passed, then the preparation and storage of new data in the cache is performed and then the data is issued in the form of a response.
If the data is available, but the cache lifetime has passed and the preparation of new data has failed, the microservice returns the last cached data and nevertheless updates the cache lifetime.

Preparation of new data:
1. The microservice downloads the web page from the url collected from the input get parameter.
2. The microservice parses an html table of about 7 columns per page (classic table/tr/td table with a few additional tags)
3. The microservice saves the result of parsing to the cache in json-format: {data:[tuple1,tuple2,...]}

Each call to the microservice should be recorded in a log file in a format similar to the apahce/nginx log, indicating:
date and timeinput parameter processing result

An excerpt from the table for example:




Series

Company
Name2
Name3

02.18
Name4
06.06
Title5
25 h 12 m

There are usually 5-20 lines per page in the table

Rational ideas are welcome.
18.10.2022 15:39



 Answers freelancers