Effective engineers will always prefer high leverage work to get done first, high leverage work is a kind of time investment, it’s to spend some time and efforts to build something that will save a lot of time in the future or prevent many human errors, one of these good investments is CI/CD pipelines.
CI/CD pipelines have never been a need only for big teams, even a single developer will need it, whatever the project it’s, however, it could have a few or many steps based on team size and needs.
Kubernetes and Cloud-native applications
I’m gonna assume that your application is a Cloud-native…
This article is the completion of another article:
Part #1: Building an API to receive raw data from clients and push it to Pub/Sub
Part #2: Running ETL Jobs to transfer and load events into BigQuery
You can check the Github Repository to run/view-code this example, full details to run this example is there, just one command!
ETL Job deploy
Events subscription could have 10 messages or 10 million messages, so the number of jobs will vary based on the number of outstanding messages!, …
One of the success keys for startups is DATA, collecting data never been a bad idea, even it might be costly in time and resources, but it’s that treasure that you will use to understand your product and move your platform to a new level of ML.
I’m gonna show you a way to handle millions of data events in a scalable way, using Serverless services provided from GCP and do have a free tier!!
This article would be split into 2 parts:
Part #1: Building an API to receive raw data from clients and push it to Pub/Sub
How many times you were surprised when your favorite app show you relevant content?
Whether you’re using Youtube, Netflix, or any other app you will notice that recommendation system is the main player in their success, but having a recommendation system could be a bit heavy for your startup, adding Machine learning algorithms like collaborative filtering / content-based filtering, after that, you need to train a model based on your big data, at the end you will end up with a static model to predict content relevance to the user.
The previous approach is hard and expensive to apply, any…
Abdul Rahman Babil
Tech Lead at Newswav | Backend and Android developer | entrepreneur | micro investor
