Data

The last ten years have seen almost a complete turnover in the tools available to a data professional. Looking at today’s Modern Data Stack, the majority of tools (dbt, Looker, Snowflake, Fivetran, Hightouch, Census) were not commercially available in 2012. Entire categories (ELT, Reverse ETL, cloud warehouses) and frameworks (data…

From Prefect 1 or another orchestration framework — At Prefect, we have an ambassador program called Club42. One of our club members recently asked: why should I migrate to Prefect 2? If you are using Prefect 1 or another orchestrator, here is a list of reasons you should consider migrating to Prefect 2. #1 Just Python Prefect 2 embraces the philosophy…

Agile, everyone talks about it, but is anyone actually doing it? No, not behind the bike sheds… Almost every company say’s “we’re agile”, and in many cases, like the peer pressured teenager, they say what they think makes them cool. Having a Jira board doesn’t mean you’re agile, neither does working in sprints. Perhaps agile development is confined to a small…

if you are planning to improve you skills and get yourself involved into some heavy technical projects , I got you , this article is for you , the next ones also , I’m going to highlight the best for you , as I mention , always , the good…

Introduction In this article we will discuss the difference between data and information. There are many ways to approach this topic. Here, we will explain the main concepts behind them and connect them to our daily lives. We will also cover Information Theory, but don’t worry if there is some math…

In 2016, before We developed Crawlo, there was a need for technologies that could monitor e-commerce sales and help businesses address the factors that drive those sales. I was often invited to board meetings at large e-commerce companies and, during these meetings, I heard many questions that couldn’t be answered…

Master them in 20 minutes. Use them every day. — I love Python’s list comprehensions and generators. They keep my code concise. They’re great one-liners for exploring and “munging” data. They’re intuitive, and once you know what you’re looking at, very easy to read. Haven’t heard of them? Read on! Think you know all the variations of this popular Python…

In 2016, the largest ever search for extraterrestrial intelligence commenced — Breakthrough Listen. This is a 10-year $100M program, funded as an act of philanthropy by tech investor Yuri Milner. Not only is the scope of the scientific effort completely unprecedented, so too is the dataset that has been made…