Tag: azure

Your all-in-one Generative AI platform: Azure AI Studio

Your all-in-one Generative AI platform: Azure AI Studio

During the last couple of months, generative AI has changed productivity, operations and customer support. Several AI providers have put a lot of emphasis on generative AI, incorporating it in their tools and offering tools such that customers can implement it inside their businesses. Besides Azure OpenAI which leverages the models from OpenAI such as […]

What orchestrator to use for your ETL jobs?

What orchestrator to use for your ETL jobs?

In today’s data-driven world, organizations are facing the ever-increasing complexity of data pipelines. The need to handle diverse data formats, sizes, sources and processing requirements can result in the need of different types of transformations blocks, ranging from SQL to PySpark to low/no-code solutions. When complex table dependencies also enter the picture, you get high, […]

Uploading Json data to an on-premise REST API using Azure Data Factory and Azure Databricks

Uploading Json data to an on-premise REST API using Azure Data Factory and Azure Databricks

The goal of this exercise is to upload some data we read from csv files, structure them as Json and upload the data to a REST API. The catch is that we are uploading this data to an on-premise API from our azure cloud. The API requires authentication via an access token, and the API […]

Connecting and Using MS Graph in Azure Data Factory

Connecting and Using MS Graph in Azure Data Factory

Companies are creating more and more data, on which they want to gain insights. One valuable source of data is data from within the company itself, from the companies’ structure. For this type of data, the MS Graph API is something to look at. The API provides a single endpoint to access all kinds of […]

Loading mechanisms – Part I

Loading mechanisms – Part I

As there are huge amounts of data available within companies, data is also moved in increasing quantities from one data storage to another for multiple reasons. As copying data can come with a longer load time and higher costs, you want to make this process as efficient as possible. Luckily, there are multiple loading mechanisms […]

Transfer learning in Spark for image recognition

Transfer learning in Spark for image recognition

Transfer learning in Spark demystified in less than 3 minutes reading Businesses that want to classify a huge set of images in batch per day can do this by leveraging the parallel processing power of PySpark and the accuracy of models trained on a huge set of images using transfer learning. Let’s first explain the […]

How ALM streamlines BI projects: Azure DevOps

How ALM streamlines BI projects: Azure DevOps

Application Lifecycle Management (ALM) refers to a (software) development process which has been setup in a governed and easy-to-manage way. ALM provides added value to the development team, project managers and the business users. While ‘ALM’ is mostly coined by pure software development projects (…written in 100% programming languages), BI projects (which are by nature […]

The Journey of attaining the Azure Data Engineer certificate

The Journey of attaining the Azure Data Engineer certificate

On February 23, 2021, Microsoft released a new beta certification exam, Exam DP-203: Data Engineering on Microsoft Azure. It is replacing the exams DP-200: Implementing an Azure Data Solution, and DP-201: Designing an Azure Data Solution. These previous exams DP-200 and DP-201 will retire on June 30, 2021. When passing the two old exams or […]

Things to consider when creating a Data Lake

Things to consider when creating a Data Lake

Have you wondered what a data lake is? What are typical use cases for this lake? How can you benefit from a data lake are? In this blog post, we will show you the added value of a data lake while pointing-out some pitfalls and best-practices. Before diving into data lakes (ba-dum-tsss), let us start […]

Architecture behind the Qatar World Cup 2022 model

Architecture behind the Qatar World Cup 2022 model

We explain the steps we took to predict the winner of the Qatar World Cup 2022 in this blog. The concepts covered include the various types of source data, the machine learning model, and the architecture used to obtain our results. Furthermore, we will go over and interpret the results in greater depth. Source data […]