Guten Tag!

Many greetings from Munich, Germany. In today’s newsletter, I want to teach you about database transactions. Sounds simple, but it’s actually a very crucial concept that we haven’t covered before.

You see, in many cases you’ll want to write to your database using transactions. Technically, you could just write to your database as-is. But this leaves you without any safety precautions in case something goes wrong.

For example, you could lose your internet connection in the middle of uploading a large dataset. If that happens your database ends up with only part of the data, and then you’re stuck trying to figure out:

• What was successfully transferred?

• What’s still only on your local machine?

These kinds of situations can easily leave your database in a broken or inconsistent state. And that’s really really frustrating to deal with.

So that’s where transactions come in. Let’s dive in.

What Are Transactions?

Transactions allow you to tell your database ahead of time

“Hey, I’m going to write something. I’ll let you know when I’m done. Don’t finalize anything until I say so.”

If everything works fine, you “commit” the transaction, and the data gets saved. If not, you “roll it back,” and it’s like nothing ever happened.

Sounds simple, right? Well, it is. You just have to use this nifty little safety feature. Your database will thank you.

How to Use Transactions in R

It’s surprisingly easy to use transactions in R when working with databases. You start a transaction with:

Then, you perform your usual data writing operations. Let’s assume we’re just going to add a few rows:

If everything goes well, you finalize the changes with:

Or if anything goes wrong, you can cancel the whole transaction, i.e. everything you did since you called dbBegin():

Why Is This Helpful?

This is especially useful when you’re working with multiple tables.

Let’s say you need to insert data into one table and then into another table that references the first table. Let’s try this:

Notice how the second call failed? That happened because I put restrictions on the second table whereas the first one doesn’t have them.

Either way, that’s when we call dbRollback(). And in order for all of this to happen automatically, we can wrap the whole procedure into a tryCatch() statement.

And with that, you’ve got a new set of tools to interact with databases safely and reliably. Use transactions whenever you write to a database, especially when multiple related operations are involved. Your future self (and your data) will thank you!

And as always, if you have any questions, or just want to reach out, feel free to contact me by replying to this mail or finding me on LinkedIn or on Bluesky.

See you next week,
Albert 👋

Enjoyed this newsletter? Here are other ways I can help you:

Data Cleaning With R Master Class

A data scientist’s guide to avoid wasting time with data cleaning. Learn 5 fundamental aspects of data cleaning, get to your insights much quicker.

data-cleaning.albert-rapp.de

Insightful Data Visualizations for "Uncreative" R Users

In this course, I teach you how to create insightful data visualizations without being a design expert.

arapp.thinkific.com/courses/insightful-data-visualizations-for-uncreative-r-users

Making Beautiful Tables with R

Tables are often boring, hard to read, and, most importantly, they don't communicate effectively. In this course, I will show you that creating beautiful tables is just a matter of learning a few important principles.

rfortherestofus.com/courses/tables

• My playlist on YouTube

• Free book: Yet Again - R + Data Science

• Free book: Beautiful tables with {gt}

• Blog posts

Guten Tag!

Many greetings from Munich, Germany. The glue_sql() function helped us last week to make our database calls safer and avoid SQL injection.

But there’s also a lot of convenience that glue_sql() offers for everyday development. So let’s check that out.

Use Curly Braces for R Variables

The first thing that’s really convenient and that you already know from last week is this: You can use curly braces {} inside your SQL string to insert R variables. This works great when the variables are just simple strings or scalar values.

And as you can see, this gives you the results you probably want:

Use Backticks for IDs

Next, let’s try to not grab all columns but only a selected one:

Well, that didn’t crash but it didn’t go as expected either. The reason for that is that you will have to tell glue_sql() that col_name (and technically also tbl_name) is an ID. You can do that with backticks inside the curly brackets.

Ahh this looks much better!

Collapse Vectors with the * Operator

If you pass a vector into a placeholder, glue_sql() by default tries to insert the whole vector resulting in multiple SQL fragments:

This is usually not what you want. Instead, you want to collapse a vector with *:

Sweet! With that you can easily get data from multiple columns:

Use IDs from different tables

Let’s imagine that our database has another table:

Then, we can make sure to reference to columns from different tables by including the table names. The Id() function helps with that:

And because we need to identify the corrsponding species columns somehow, we have to reference to the corresponding table names as well.

Now, we could easily wrap all of these IDs into backticks and everything works out nicely in assembled SQL code. Here’s an example from the glue_sql() docs for that:

And there you have it: glue_sql() will make your life as an R developer much easier. Beyond just protecting you from SQL injection, it helps you write cleaner, more flexible, and maintainable database queries from within R.

And as always, if you have any questions, or just want to reach out, feel free to contact me by replying to this mail or finding me on LinkedIn or on Bluesky.

See you next week,
Albert 👋

Enjoyed this newsletter? Here are other ways I can help you:

Data Cleaning With R Master Class

A data scientist’s guide to avoid wasting time with data cleaning. Learn 5 fundamental aspects of data cleaning, get to your insights much quicker.

data-cleaning.albert-rapp.de

Insightful Data Visualizations for "Uncreative" R Users

In this course, I teach you how to create insightful data visualizations without being a design expert.

arapp.thinkific.com/courses/insightful-data-visualizations-for-uncreative-r-users

Making Beautiful Tables with R

Tables are often boring, hard to read, and, most importantly, they don't communicate effectively. In this course, I will show you that creating beautiful tables is just a matter of learning a few important principles.

rfortherestofus.com/courses/tables

• My playlist on YouTube

• Free book: Yet Again - R + Data Science

• Free book: Beautiful tables with {gt}

• Blog posts

Guten Tag!

Many greetings from Munich, Germany. In the last couple of newsletters, we talked about SQL databases. And there’s no way we can talk about accessing SQL databases via R code without mentioning SQL injection.

This is a nasty thing that can leak data or even destroy your database if you allow people to interact with it unsafely. And often, that’s exactly what you want (the interaction, not the unsafe part 😆). Here’s a common scenario for that:

• Users interact with a Shiny dashboard to retrieve specific data from a database.

• Based on the inputs, you execute some SQL.

• Your dashboard displays the new data.

So letting users interact with a database is not bad in itself. It’s only bad when you do it uncontrolled. That’s when this bad SQL injection can occur.

But here’s the good news: There’s an easy way to avoid damage. And even better: This method also makes it much easier to assemble SQL code in R.

Let me show you how that works.

SQL Injection via R Variables

Let’s talk about how you might encounter SQL injection in the first place.

Let’s imagine that you want to retrieve data about a specific penguin species from a table in your database. One common (but unsafe) way to do this is to paste together SQL code manually with R variables:

While this works, it’s insecure. If a user-supplied value like species contains malicious SQL code, it could complete your SQL snippet and do something dangerous (e.g. extracting all of your confidential data). This is SQL injection.

And in this specific scenario, here’s what that would look like. Let’s first set up our data base from last time:

And then we can stick in some malicious code into the species variable.

Here, a little trick was used to close the “WHERE” statement with a ' followed by an or-statement that is always true. That way, we will never actually filter the data.

Why Would That Happen?

Now, you might wonder: Why would my variable ever be filled with something malicious?

Well, consider a Shiny app that allows users to filter database results. If you’re not careful, users can input harmful values into the fields that populate those SQL variables. If someone with bad intentions figures out you’re pasting user input into SQL strings, they could inject destructive SQL.

And they don’t even have to do that manually. There are automated tools that try out loads of combinations in the hopes that at some point some SQL injection works.

The Solution: glue_sql() from {glue}

Here’s where {glue} and its glue_sql() function come in. Using glue_sql() makes assembling SQL both easier and safer. You still write your SQL code in text form, but you use curly braces ({}) to insert variables.

Most importantly, glue_sql() handles SQL escaping for you to prevent SQL injection. Here’s an example:

So in the standard case everything works like you’d expect. And in case there’s the same malicious code from before inside the species variable?

Here, glue_sql() will put everything into quotes. That way, a user cannot stick in something to close the initial quotation mark. And the resuls would be empty:

Thus, this is easy to write and secure.

Fantastic! We’ve just seen how glue_sql() helps you write safer SQL code. But there’s more! In the next newsletter, we’ll explore additional features of glue_sql() that make it even more powerful.

And as always, if you have any questions, or just want to reach out, feel free to contact me by replying to this mail or finding me on LinkedIn or on Bluesky.

See you next week,
Albert 👋

Enjoyed this newsletter? Here are other ways I can help you:

Data Cleaning With R Master Class

A data scientist’s guide to avoid wasting time with data cleaning. Learn 5 fundamental aspects of data cleaning, get to your insights much quicker.

data-cleaning.albert-rapp.de

Insightful Data Visualizations for "Uncreative" R Users

In this course, I teach you how to create insightful data visualizations without being a design expert.

arapp.thinkific.com/courses/insightful-data-visualizations-for-uncreative-r-users

Making Beautiful Tables with R

Tables are often boring, hard to read, and, most importantly, they don't communicate effectively. In this course, I will show you that creating beautiful tables is just a matter of learning a few important principles.

rfortherestofus.com/courses/tables

• My playlist on YouTube

• Free book: Yet Again - R + Data Science

• Free book: Beautiful tables with {gt}

• Blog posts

Guten Tag!

Many greetings from Munich, Germany. Last week, I showed you how to connect to databases with R. And I also mentioned that it’s helpful to write a bit more SQL code than just grabbing all the data and processing it locally. This might sound like a lot of effort, though. So here’s the good news:

If you’re already using dplyr for data wrangling, you can get the best of both worlds. You can write dplyr code and let R translate that into SQL so that time-saving and resource-efficient calculations happen in the database. Then, you only retrieve the results.

So in today’s newsletter I’ll show you how that works. Let’s go!

Getting Started with {dbplyr}

First things first: you’ll need to install the {dbplyr} package. This package extends dplyr so it can translate R code into SQL syntax for supported databases.

To get started, establish a connection to your database using DBI::dbConnect(), just like before. This part hasn’t changed.

Working with Tables Using {dbplyr}

Now you can use {dbplyr} functions to interact with the database using familiar dplyr verbs. For example, in our example database, you can copy an R dataset like palmerpenguins::penguins into a database table using copy_to():

Fun fact: Notice that I never loaded {dbplyr} 😮 {dplyr} will detect when you want to use a {dbplyr} function.

Once the data is in the database, you can reference it with tbl(). This will connect to the table in the database (which is why the con variable is needed.)

Notice how the output almost looks like a regular tibble. The only difference is that it shows only a preview and not the full dataset. And in case you’re wondering: That’s a good thing. You never know how large the database is and often a preview will suffice.

Click to Support 👇️ 

Hey friends 👋 I’m experimenting with ads in my newsletter to finance all the free stuff I give away. If you enjoy this newsletter and want to support it, a click on the the ad below would be a super simple but effective way to do so. 🤗 

No need to sign up. A click is enough support 🥳 

---

You’ve never experienced business news like this.

Morning Brew delivers business news the way busy professionals want it — quick, clear, and written like a human.

No jargon. No endless paragraphs. Just the day’s most important stories, with a dash of personality that makes them surprisingly fun to read.

No matter your industry, Morning Brew’s daily email keeps you up to speed on the news shaping your career and life—in a way you’ll actually enjoy.

Best part? It’s 100% free. Sign up in 15 seconds, and if you end up missing the long, drawn-out articles of traditional business media, you can always go back.

Check it out

---

Alright, now back to the good stuff.

Writing SQL Through dplyr Verbs

Now that you have a table-like object, you can use familiar dplyr verbs to generate SQL queries. For example:

Again, this only shows a preview. To pull the full result set from the database, use collect().

Seeing the Generated SQL

And if you’re ever wondering how {dbplyr} translated your dplyr code into SQL, you can inspect it using show_query().

Also, this is a great way to learn more about SQL by seeing how familiar R code maps to SQL commands.

Nice. We’ve learned a simple way to interact with databases using familiar syntax. However, using {dbplyr} doesn’t mean you’ll never need to write raw SQL again.

It helps you get started quickly and efficiently, though. Next week, I’ll show you how to combine {dbplyr} with raw SQL for more advanced database workflows.

And as always, if you have any questions, or just want to reach out, feel free to contact me by replying to this mail or finding me on LinkedIn or on Bluesky.

See you next week,
Albert 👋

Enjoyed this newsletter? Here are other ways I can help you:

Data Cleaning With R Master Class

A data scientist’s guide to avoid wasting time with data cleaning. Learn 5 fundamental aspects of data cleaning, get to your insights much quicker.

data-cleaning.albert-rapp.de

Insightful Data Visualizations for "Uncreative" R Users

In this course, I teach you how to create insightful data visualizations without being a design expert.

arapp.thinkific.com/courses/insightful-data-visualizations-for-uncreative-r-users

Making Beautiful Tables with R

Tables are often boring, hard to read, and, most importantly, they don't communicate effectively. In this course, I will show you that creating beautiful tables is just a matter of learning a few important principles.

rfortherestofus.com/courses/tables

• My playlist on YouTube

• Free book: Yet Again - R + Data Science

• Free book: Beautiful tables with {gt}

• Blog posts

Guten Tag!

Many greetings from Munich, Germany. When you’re working in a company, chances are you won’t be handed a simple CSV file to work with. Instead, you’ll likely need to grab your data from a company-wide SQL database hosted in the cloud or on internal servers.

So in today’s newsletter, I’ll show you how to connect to these databases using R.

What The Heck is a Driver?

First, we need a so-called driver. In simple terms, this is the tool that handles the connection between your computer and the database. Think of it as a translator that often comes in one of two flavors:

• ODBC drivers: These drivers work with the open database connection format

• JDBC drivers: These drivers use Java (that’s what the “J” stands for in JDBC)

Here, we’ll use the {odbc} package to go down the ODBC route.

Which Driver Do You Need?

So now that we know that we want ODBC drivers, we also need to decide which specific driver we need. You see, this kind of thing depends on the database your or your company uses. For example it could be an Oracle or PostgreSQL database that will need an Oracle or PostgreSQL driver

Typically, your tech department will tell you what you need. Your job is only to understand the big picture that I’m trying to explain here.

Example: Setting Up an SQLite Database

For today’s example, we’ll simulate a database using SQLite, which supports in-memory databases. This means that we can simply spin up a temporary database locally on our computer.

After having installed a SQLite driver for your computer, you can set up a connection from R to a temporary database using {DBI} and {odbc}:

Whooah. That’s a lot of function arguments. Once again, the connection details (like database name, user, password and port) is something that your tech department should have ready for you.

Creating and Populating a Table

So now that we have a database connection stored in a variable called con, we can create a table in our SQLite database. Here, we’ll use the famous Palmer Penguins dataset:

Querying Data from the Database

To fetch data, you’ll use dbGetQuery() from the {DBI} package. Here’s how to select all rows using SQL code:

Alternatively, you can (and probably should) use a bit more SQL to filter down the data that you actually want.

Why Use SQL for Filtering?

And you may wonder: “Why should I invest time into learning SQL when I can just grab the full data using the snippet you’ve just shown?“ And indeed, you could download the entire table and filter it inside of R. But filtering directly in the database is often much faster because:

• SQL databases are optimized for these operations

• You avoid transferring large datasets over the network

Learning a few basic SQL keywords can go a long way in improving your data workflow.

Nice! With that we covered:

• The basics of database connections in R

• The difference between ODBC and JDBC

• How to set up and query a database using {DBI} and {odbc}

Armed with these skills, you’ll hopefully be ready to connect to real company databases and work with large-scale data. As always, if you have any questions, or just want to reach out, feel free to contact me by replying to this mail or finding me on LinkedIn or on Bluesky.

See you next week,
Albert 👋

Enjoyed this newsletter? Here are other ways I can help you:

Data Cleaning With R Master Class

A data scientist’s guide to avoid wasting time with data cleaning. Learn 5 fundamental aspects of data cleaning, get to your insights much quicker.

data-cleaning.albert-rapp.de

Insightful Data Visualizations for "Uncreative" R Users

In this course, I teach you how to create insightful data visualizations without being a design expert.

arapp.thinkific.com/courses/insightful-data-visualizations-for-uncreative-r-users

Making Beautiful Tables with R

Tables are often boring, hard to read, and, most importantly, they don't communicate effectively. In this course, I will show you that creating beautiful tables is just a matter of learning a few important principles.

rfortherestofus.com/courses/tables

• My playlist on YouTube

• Free book: Yet Again - R + Data Science

• Free book: Beautiful tables with {gt}

• Blog posts

Guten Tag!

Many greetings from Munich, Germany. Environment variables control system-wide or session-wide settings inside your R session. This sounds incredibly boring. And, honestly, it kind of is.

But it can feel complicated to R users who haven’t encountered them before. When you do bump into environment variables “in the wild,” they can seem unnecessarily tricky.

That’s why, in today’s newsletter, I’ll cover a bit of the mundane stuff. That is, I’ll show you how to set and get environment variables in R. I know: Exciting! So let’s dive in.

When Do Environment Variables Matter?

Let’s quickly look at a couple of real-world examples where environment variables show up in R workflows:

Using APIs (like OpenAI)

If you’re working with APIs, such as using the {ellmer} package to connect to ChatGPT or other AI services, you’ll need to store your API keys as environment variables.

For example, to use OpenAI’s API, you need to specify the environment variable OPENAI_API_KEY. This keeps your credentials secure and avoids hard-coding sensitive infos into your scripts.

Deploying Shiny Apps

If you’re deploying a Shiny app, you might want to keep different settings for:

• Production (the live version of your app that your users see)

• Development (the version of the app that you as a developer see and use to test new features)

Environment variables let you switch configurations easily depending on your deployment stage (without editing the core code.)

So with the examples out of the way, let’s dive into three popular methods for setting environment variables.

Method 1: .Rprofile (My Favorite)

The .Rprofile file allows you to run R code automatically at the start of each R session. If your project doesn’t have such a file, just create a text file and name it .Rprofile. And to set an environment variable, you can use Sys.setenv() inside the .Rprofile file:

🚨 NOTE: Only use Sys.setenv() inside your .Rprofile. If you put it in a regular script, it might get saved to your R history file. And if that ever gets pushed to a public repo, your API key could be compromised.

Method 2: .Renviron

The .Renviron file is simpler. Unlike .Rprofile, it cannot execute R code as it only holds key-value pairs. But just like .RProfile, these environment variables are always loaded at the start of an R session.

The file itself always follows the syntax key=value. Here’s an example:

Method 3: .env (via {dotenv})

The .env file is popular in many programming languages (like Python). A .env file looks just like .Renviron. The difference is only that you load the .env file manually. In R, you can use the {dotenv} package to do that.

This method is especially handy if you want to share environment variables across R and Python.

There you have it. Three ways to work with environment variables in R:

1️⃣ .Rprofile (using Sys.setenv())
2️⃣ .Renviron (key-value only)
3️⃣ .env (loaded via {dotenv})

Sure, it’s not the flashiest topic but it’s super important once you move into production-like environments or need to store secrets. As always, if you have any questions, or just want to reach out, feel free to contact me by replying to this mail or finding me on LinkedIn or on Bluesky.

See you next week,
Albert 👋

Enjoyed this newsletter? Here are other ways I can help you:

Data Cleaning With R Master Class

A data scientist’s guide to avoid wasting time with data cleaning. Learn 5 fundamental aspects of data cleaning, get to your insights much quicker.

data-cleaning.albert-rapp.de

Insightful Data Visualizations for "Uncreative" R Users

In this course, I teach you how to create insightful data visualizations without being a design expert.

arapp.thinkific.com/courses/insightful-data-visualizations-for-uncreative-r-users

Making Beautiful Tables with R

Tables are often boring, hard to read, and, most importantly, they don't communicate effectively. In this course, I will show you that creating beautiful tables is just a matter of learning a few important principles.

rfortherestofus.com/courses/tables

• My playlist on YouTube

• Free book: Yet Again - R + Data Science

• Free book: Beautiful tables with {gt}

• Blog posts

Guten Tag!

Many greetings from Munich, Germany. In today’s newsletter, I’ll show you how to integrate multiple parametric report sections into a single report. You see, instead of generating separate reports for each dataset, you can include multiple sections where each sections uses different data.

This could look something like this:

• 📄 General intro text (applies to all datasets)

• 🐟 Section A (Dataset A)

• 🐦 Section B (Dataset B)

• 🦋 Section C (Dataset C)

• ✅ General conclusion

So let’s see how to build this.

Step 1: Start with a Parametric Report

Begin with a standard parametric report, just like you normally would.

For example, let’s say you have the Quarto file from last time that generates a report for a single species. This was driven by a species and a color parameter in the YAML header:

This report works fine on its own, but we want to reuse the code and text within the report multiple times in a single document.

Step 2: Create a Main Quarto File

Next, create a new Quarto file that will act as the main document. This file will include first some general intro text, followed by multiple “includes” of your parametric report. For example, the document could start like this:

Step 3: Use include Snippets

Quarto has a special feature that lets you include other Quarto files using the following syntax:

Here, this will include text and code chunks from the Quarto file species_report.qmd. That way, you could modify your main Quarto file like this:

But if you render this now, you’ll hit an error that will say that params is not defined. If you recall, params is the object that we use inside the species Quarto file.

Normally, this object is created from the parameters that are specifies inside the YAML header. But here this doesn’t work because includes don’t copy the YAML header from the included file.

Step 4: Define params in the Master File

To solve this, define the params object manually in a code chunk before each include. For example, your report could look like this:

That way, the copied-and-pasted code from species_report.qmd will work because it is preceded by a code chunk that defines params.

Step 5: Repeat for Multiple Datasets

Now, you can repeat the same process to include multiple sections, each with a different dataset:

This reuses the same species report over and over, but with different params each time. Super efficient, isn’t it?

This approach allows you to

• reuse the same report structure,

• render multiple sections with different data, and

• produce a single, polished document

It’s a powerful pattern when working with multiple datasets that need similar analysis! Pretty neat, right? As always, if you have any questions, or just want to reach out, feel free to contact me by replying to this mail or finding me on LinkedIn or on Bluesky.

See you next week,
Albert 👋

Enjoyed this newsletter? Here are other ways I can help you:

Data Cleaning With R Master Class

A data scientist’s guide to avoid wasting time with data cleaning. Learn 5 fundamental aspects of data cleaning, get to your insights much quicker.

data-cleaning.albert-rapp.de

Insightful Data Visualizations for "Uncreative" R Users

In this course, I teach you how to create insightful data visualizations without being a design expert.

arapp.thinkific.com/courses/insightful-data-visualizations-for-uncreative-r-users

Making Beautiful Tables with R

Tables are often boring, hard to read, and, most importantly, they don't communicate effectively. In this course, I will show you that creating beautiful tables is just a matter of learning a few important principles.

rfortherestofus.com/courses/tables

• My playlist on YouTube

• Free book: Yet Again - R + Data Science

• Free book: Beautiful tables with {gt}

• Blog posts

Guten Tag!

Many greetings from Munich, Germany. In this week’s newsletter, we’re continuing our journey with Typst + Quarto. As promised, we’re taking our PDF reports to the next level with some visual polish.

Today, I want to show you how to:

1. Add a vertical color bar along the left margin of every page

2. Wrap your main title in a colored box and shift it slightly to the left for a bold visual effect

These small touches can improve the look and feel of your report. And once you know how these things work, you can work out how to add more visual elements with Typst.

Add a Vertical Color Bar to the Left Margin

Let’s start by giving all pages a distinctive look using a colored stripe along the left-hand side. To do this, we modify the set page() section of our Typst template. You know, this part here:

Here, we can use the background property with the place() function to overlay a rectangle:

And with this simple change, we’ve added a vertical splash of color to every page.

Style the Main Title with a Colored Box

Next, let us give our report title more visual weight. Instead of plain text, we’ll wrap it in a colored box. We do so by wrapping our text() function into a box() function. That way, our previous pages specification

becomes

Now, our document looks like this:

Shift the title box

Finally, we can shift the title box slightly to the left by adding a bit of negative horizontal space.

With just a few small tweaks, we now have a PDF report that is not only parameterized with different inputs but it also looks polished and professional.

And as always, if you have any questions, or just want to reach out, feel free to contact me by replying to this mail or finding me on LinkedIn or on Bluesky.

See you next week,
Albert 👋

Enjoyed this newsletter? Here are other ways I can help you:

Data Cleaning With R Master Class

A data scientist’s guide to avoid wasting time with data cleaning. Learn 5 fundamental aspects of data cleaning, get to your insights much quicker.

data-cleaning.albert-rapp.de

Insightful Data Visualizations for "Uncreative" R Users

In this course, I teach you how to create insightful data visualizations without being a design expert.

arapp.thinkific.com/courses/insightful-data-visualizations-for-uncreative-r-users

Making Beautiful Tables with R

Tables are often boring, hard to read, and, most importantly, they don't communicate effectively. In this course, I will show you that creating beautiful tables is just a matter of learning a few important principles.

rfortherestofus.com/courses/tables

• My playlist on YouTube

• Free book: Yet Again - R + Data Science

• Free book: Beautiful tables with {gt}

• Blog posts

Guten Tag!

Many greetings from Munich, Germany. Have you ever heard of the Typst project? It is a lightweight, Markdown-inspired language designed for creating beautiful, customized PDF documents with minimal effort.

Basically, it is supposed to be like LaTeX but muuuch easier. And the coolest part? You can use Typst from within Quarto to create stunning PDF reports.

So in today’s newsletter, I’ll walk you through how to style our PDF document from last week. Let’s dive in

Set Typst as Your PDF Format

To start using Typst from Quarto, go into your .qmd file and specify typst as the output format in the YAML header:

If you render this now, you’ll get a similar PDF output as before.

Add Custom Styling with Typst Templates

To make our PDF look good, we’ll use template partials provided by Typst. There are two key files that we need to mention in the YAML header for that:

Both are plain text files that you can create by just taking a .txt file and changing the extension to .typ. So the more important question is: “How do we fill these files?” Let’s go through both files.

Filling typst-show.typ

This file defines the variables that are passed into the layout, such as the title, author, and parameters from your Quarto YAML. You see, what you you define in the Quarto YAML header isn’t automatically known to Typst. That’s why you need to transfer it. For that you use the #show function from Typst.

There, you specify that your “article” layout (that’s the default name of the used layout) is equipped with a set of variables.

The syntax takes some getting used to. Basically, you always write

Yeah, I know. As I said, it takes some getting used to. And for drilling down nested YAML structures, you can just use a dot symbol (.). With that we know enough Typst code to transfer title, author and params.species from our Quarto YAML header to Typst.

Filling typst-template.typ

Nice. One file down, one more to go.

In the typst-template.typ file, we define the actual article layout of our PDF report. For that, we use the #let function.

Inside the function body (indicated by {}), we can now set attributes of

• document (contains metadata),

• text (defines text properties of the document), and

• page (defines the properties of every page in the document).

Have a look at the code and see if you can make out what it does. I like to think that it’s somewhat manageable to piece together what it does. But don’t worry. I’ll explain afterwards anyway.

So here, we have used the set statement to specify properties of the document, text and page objects. There, we’ve set some of the objects attributes (like title, font or paper) to values that were

• either hard-coded (like "Source Sans Pro" or the margins) or

• filled using arguments of the #let() function (like author or paper).

Add a page

So far so good. We’ve set a couple of things. But we haven’t added any content to the layout yet. Let’s change that.

Let’s insert a page using the page() function. In there, we can make sure that everything is left-aligned and starts at the top of the page using align(). The syntax for that is

Add content

See that line that says “page content goes” here? That’s where we can place the content of the page. Here, let’s just try to use the body variable from the article() function to insert everything that is inside our Quarto document.

That should be a valid layout. Let’s try to render this. Here’s what we’ll get:

Oh no. That’s a bit dumb. Instead of understanding body as variable, Typst things that we actually want to use a literal text. But we have used other variables just like that before, haven’t we? What’s going on?

Markdown vs evaluation context

You see, we are currently filling the align() function inside the page() function with content. This content is inside brackets [] where it is evaluated as Markdown. Not in any functional kind of way. And the magical signal for Typst to switch to an evaluated context is #.

That’s also why the let() function to define the article layout started with #. Otherwise, Typst would think that our file is just filled with plain text and won’t evaluate anything. And the same has do be done now that we are inside a Markdown content block inside the brackets of align(). So, let’s try this:

Ahh much better. There’s all of our content. And notice how everything fits on one page now.

Add our parameters

Similarly, we can add our parameters using the text() function. The syntax is the same as with align():

• First, we fill text() with arguments like text(size:1.6em).

• Then, we create actual content inside the brackets like `text(size: 1.6em)[content].

• And finally we have to use # inside the Markdown content blocks to use functions and variables.

This will give us this terrible look.

Let’s add some vertical space with the v() function.

And then we have way more space:

Ahh much better. Let that sink in for now. Next week, I’m showing you how to add colors to our designs.

And as always, if you have any questions, or just want to reach out, feel free to contact me by replying to this mail or finding me on LinkedIn or on Bluesky.

See you next week,
Albert 👋

Enjoyed this newsletter? Here are other ways I can help you:

Data Cleaning With R Master Class

A data scientist’s guide to avoid wasting time with data cleaning. Learn 5 fundamental aspects of data cleaning, get to your insights much quicker.

data-cleaning.albert-rapp.de

Insightful Data Visualizations for "Uncreative" R Users

In this course, I teach you how to create insightful data visualizations without being a design expert.

arapp.thinkific.com/courses/insightful-data-visualizations-for-uncreative-r-users

Making Beautiful Tables with R

Tables are often boring, hard to read, and, most importantly, they don't communicate effectively. In this course, I will show you that creating beautiful tables is just a matter of learning a few important principles.

rfortherestofus.com/courses/tables

• My playlist on YouTube

• Free book: Yet Again - R + Data Science

• Free book: Beautiful tables with {gt}

• Blog posts

Guten Tag!

Many greetings from Munich, Germany. Parameterized reports are a powerful but often underutilized strategy for communicating data to multiple stakeholders.

Let’s say you have several departments in your organization. A common instinct is to build a dashboard where users can click around to see the data for their department.

But what if, instead, you could automatically generate one customized report per department and send those out directly? In today’s newsletter, I’ll show you how to do just that using Quarto.

Create a Basic Report Template

First, we need a sample report. We’ll use the palmerpenguins dataset to simulate a situation where each penguin species represents a department.

This will give us a PDF output like this:

Add Parameters to the YAML Header

Here, we’ve used the "Adelie" species to act a placeholder. Next, we have to make sure that we can easily replace the species in all instances of the text and code. To do that, we define a params section in the YAML header:

Now, wherever we want to reference species in the document, we use params$species. This allows the content of the report to adapt to whatever value we plug into the parameter. It works in code chunks:

And it works in inline text as well:

Now we can try to change "Adelie" to "Gentoo" and re-render the report. You’ll see the whole report adjusts accordingly (most notably seen by the plot change.)

Automate Rendering for Multiple Parameters

Now the real power kicks in. We want to render one report per species, or in your real-world scenario, one report per department. To do that, we have to create a separate R script that uses the quarto_render() function from the {quarto} package:

This script loops over all species and renders a separate PDF report for each one.

With just a few lines of configuration and code, you now have a scalable way to generate customized PDF reports for any number of stakeholders or categories. Whether you’re reporting by department, product line, region, or team, parameterized Quarto documents make it simple and powerful.

Next week, we’re going to make that report prettier using the Typst-PDF format. And as always, if you have any questions, or just want to reach out, feel free to contact me by replying to this mail or finding me on LinkedIn or on Bluesky.

See you next week,
Albert 👋

Enjoyed this newsletter? Here are other ways I can help you:

Data Cleaning With R Master Class

A data scientist’s guide to avoid wasting time with data cleaning. Learn 5 fundamental aspects of data cleaning, get to your insights much quicker.

data-cleaning.albert-rapp.de

Insightful Data Visualizations for "Uncreative" R Users

In this course, I teach you how to create insightful data visualizations without being a design expert.

arapp.thinkific.com/courses/insightful-data-visualizations-for-uncreative-r-users

Making Beautiful Tables with R

Tables are often boring, hard to read, and, most importantly, they don't communicate effectively. In this course, I will show you that creating beautiful tables is just a matter of learning a few important principles.

rfortherestofus.com/courses/tables

• My playlist on YouTube

• Free book: Yet Again - R + Data Science

• Free book: Beautiful tables with {gt}

• Blog posts

Guten Tag!

Many greetings from Munich, Germany. Have you heard about brand.yml? It’s one of Posit’s newest and most exciting additions to their product lineup. With brand.yml, you can define a unified theme that works seamlessly across

• Quarto documents,

• Shiny for R apps, and

• Shiny for Python apps.

In today’s newsletter, I’ll walk you through what brand.yml is and how to use it to bring consistent styling to all your projects.

What’s the idea?

The idea behind brand.yml is simple but powerful:

1. You define your primary color, secondary color, fonts, logo, and other branding settings in a single brand.yml file.

2. You can then reuse that same configuration across all your data products. No need to redefine styles in each individual project.

This makes it easier to

• maintain consistent branding,

• reuse themes across projects, and

• share themes across teams.

How does this work?

You’re probably now wondering: What does a brand.yml file actually look like? Well, let me show you. It’s just a plain text YAML file. Here’s an example from the official docs:

As you can see, it looks pretty simple. But this simple file becomes the central definition of your app or document’s look and feel.

How to Use brand.yml?

The way you use this file depends on your toolchain. Here’s a breakdown for each use case:

📄 In Quarto

First, you need to save the branding content into a file called _brand.yml. And to apply the brand in a Quarto document you just have to add the brand attribute to the YAML header and fill it with the path to your yml-file. This could look something like this:

It doesn’t get any easier than that. And if you want to use the specified branding options (e.g. colors) in a particular setting (like Typst PDFs), then you can check out the dedicated Quarto documentation on that.

💻 In Shiny for R

In Shiny for R, brand.yml is integrated via the fantastic {bslib} package.
You can use it in your UI definition like so:

🐍 In Shiny for Python

You’ll need the latest version of Py-Shiny installed with the theme extra.

Once you’ve got that, you can use brand.yml like this:

And if you’re using Py-Shiny Express, the usage is slightly different. There, you’ll have to configure the page_options argument in the UI object.

I think brand.yml is a fantastic idea. It simplifies theming, supports consistency, and works beautifully across multiple frameworks. If you’re building out a data product pipeline that includes reports and interactive apps, this tool will save you time and help keep everything looking polished and on-brand.

Let me know what you think about brand.yml. And as always, if you have any questions, or just want to reach out, feel free to contact me by replying to this mail or finding me on LinkedIn or on Bluesky.

See you next week,
Albert 👋

Enjoyed this newsletter? Here are other ways I can help you:

Data Cleaning With R Master Class

A data scientist’s guide to avoid wasting time with data cleaning. Learn 5 fundamental aspects of data cleaning, get to your insights much quicker.

data-cleaning.albert-rapp.de

Insightful Data Visualizations for "Uncreative" R Users

In this course, I teach you how to create insightful data visualizations without being a design expert.

arapp.thinkific.com/courses/insightful-data-visualizations-for-uncreative-r-users

Making Beautiful Tables with R

Tables are often boring, hard to read, and, most importantly, they don't communicate effectively. In this course, I will show you that creating beautiful tables is just a matter of learning a few important principles.

rfortherestofus.com/courses/tables

• My playlist on YouTube

• Free book: Yet Again - R + Data Science

• Free book: Beautiful tables with {gt}

• Blog posts

Guten Tag!

Many greetings from Munich, Germany. Recently, I’ve been playing around with the new Positron IDE, and I must say: I really enjoy it. It still has some small bugs but overall it’s great. More importantly, it’s packed with features tailored for data science workflows.

In this week’s newsletter, I want to share my favorite features, settings, and shortcuts in Positron. That way, you can get a sense of why it might be a better fit than VSCode for your R or Python work.

Fixed Console for R and Python

One of the standout features in Positron is the fixed console for R and Python that never goes away. Unlike VSCode, there’s always a R or Python console that you can use for quick iterations. Also, you can easily switch between languages and even between different versions of R and Python.

And regardless of what you use, there’s a dedicated console that always receives your commands when you press Ctrl + Enter in the editor. This makes it far more reliable than what I experienced with VS Code, where the console sometimes wouldn’t respond or would detach from the current session.

And to make it perfectly clear, just for this feature alone, I’d already recommend Positron over VS Code.

Integrated Help Pane (F1 to View Function Docs)

Just like in RStudio, Positron lets you press F1 on any function in your script to open the Help page.

No more switching to a browser or dealing with limited pop-ups like in VS Code. This feature is built-in and persistent. It’s especially handy when you have to refer to arguments or function references on the fly.

The only issue that I have is that the formatting in the help windows isn’t always great. In particular, if you increase your editor font size. In that case, you will have a large code font and regular text font inside the help panel.

That’s not particularly great but I hope the kind people behind Positron are already on it. For now a workaround is to increase the window size instead if you want to have larger fonts.

Side-by-Side Layout (Just Like RStudio)

I love Positron’s layout customization. When you download the newest version, there’s a button in the top-right corner to switch layouts.

I always go with the side-by-side layout, similar to what I used in RStudio:

• Left: Script or Quarto file

• Right: Console, plots, and environment

This makes transitioning from my previous RStudio setup feel completely natural.

Zen Mode for Maximum Focus

Positron has a fantastic Zen mode that lets you minimize distractions and maximize space. That way, only the essential UI things are maximized. Here’s how it looks as I’m writing this newsletter.

Depending on what I need at the moment, I can toggle between:

• Code only

• Code + Console

• Code + Console + File Explorer

Familiar Shortcuts from RStudio

Positron supports many of your favorite RStudio shortcuts:

• Ctrl + Shift + M → Insert pipe (|>)

• Ctrl + Shift + K → Render Quarto document

I’ve also customized a few additional shortcuts to toggle the sidebars for Zen mode (like I’ve just described). Or to jump between console and editor.

And in case this feels intimidating, there’s also a UI that you can use to tweak these things. It’s just easier to share the settings with you this way.

Use the Command Palette (Ctrl + Shift + P)

The Command Palette is your best friend in Positron. Press Ctrl + Shift + P to access any command or setting.

Want to enter Zen Mode? Type “Zen” and the relevant action will pop up. You’ll also see the keyboard shortcut for it, helping you memorize it over time

Pro tip: If you delete the arrow > that shows up in the palette, it becomes a fuzzy finder for files and objects in your workspace.

Projects? Nope. Directories Instead.

Positron doesn’t use “projects” the way RStudio does. Instead, it uses directories. If you had an RStudio project before, just open the directory in Positron using: Ctrl + K followed by Ctrl + O 

That directory becomes your workspace root. And everything inside it is visible in the file explorer. Pretty much like RStudio projects.

Customize Everything with settings.json

Like VSCode, Positron is fully customizable via the settings.json file. Open it via the Command Palette and throw in any setting you want. Here are some of my favorites:

And just like with the keybindings.json, there’s also a UI that you can use.

Positron is a joy to use. It brings the best of VSCode’s extensibility and combines it with RStudio’s comfort, especially for data scientists. Let me know if you’re using it already and what your favorite tweaks are. I’d love to hear what’s working for you!

As always, if you have any questions, or just want to reach out, feel free to contact me by replying to this mail or finding me on LinkedIn or on Bluesky.

See you next week,
Albert 👋

Enjoyed this newsletter? Here are other ways I can help you:

Data Cleaning With R Master Class

A data scientist’s guide to avoid wasting time with data cleaning. Learn 5 fundamental aspects of data cleaning, get to your insights much quicker.

data-cleaning.albert-rapp.de

Insightful Data Visualizations for "Uncreative" R Users

In this course, I teach you how to create insightful data visualizations without being a design expert.

arapp.thinkific.com/courses/insightful-data-visualizations-for-uncreative-r-users

Making Beautiful Tables with R

Tables are often boring, hard to read, and, most importantly, they don't communicate effectively. In this course, I will show you that creating beautiful tables is just a matter of learning a few important principles.

rfortherestofus.com/courses/tables

• My playlist on YouTube

• Free book: Yet Again - R + Data Science

• Free book: Beautiful tables with {gt}

• Blog posts

Guten Tag!

Many greetings from Munich, Germany. In today’s newsletter, I want to show you how you can combine the {pins} package with GitHub Actions. That way, you can automate the process of downloading data from the internet and saving it as a time series.

The workflow is pretty similar to one of our previous newsletter on GitHub Actions. So if you’ve read that newsletter, then you should have no problem following along.

A repo for our project

The first thing we need is a new project for our data extraction endeavor. But let’s not start completely from scratch. Let’s use our “gh_actions_test” repo. It already has an {renv} environment for version control set up. That way, we don’t have to worry about that.

Add pins to {renv}

Now we can add {pins} to our {renv} environment. Simply call the install() command.

Then, we can create a new R-Script pins-demo.R in which we’ll actually use {pins}. For now, this should only contain this code:

That way, {renv} lets us write the {pins} package to its lock file. If you recall, this is the thing we have to restore at the beginning of every GH workflow. Now make sure that

• you can execute this code in your project, i.e. you have AWS credentials set,

• that you have your .Rprofile file removed from the git repo and added it to .gitignore (because you don’t want to share your keys),

• and that you saved the packages to the lock file via renv::snapshot().

Alright, cool. So with that we need a new GH actions workflow file.

Base GitHub Actions Workflow

Let’s start out with .yml workflow file from our last GitHub Actions tutorial. That’s a good place to start I’d say.

In this project, we don’t need anything from Quarto and we also don’t need to commit to Github, so we can remove those steps.

Unfortunately, R isn’t contained in the ubuntu-latest machine from GitHub anymore. That’s why we have to add R ourselves. But that’s super easy to do. We only have to use one of the R containers provided from the Rocker Project.

And with that we are once again ready to use R automations.

Extract Data

Next, we need an R function that fetches data from an external source. This could be via

• API calls,

• web scrapings, or

• any other data retrieval method.

To keep things simple, we’ll simulate it with a function that just returns a random value:

Check if pin is available

In the next step, we want to write this data to a pin that is stored in our board in S3. For that, we first need to check if our pin is even available. If not, we simply write to that pin without doing anything else.

Grab previous data

If the pin is already available, then we should

• grab the already saved data,

• append our new data to it, and

• write that new complete data to the same pin.

And that’s it. {pins} will do all the data versioning for us. So overall, our pins-demo.R script is only a short little file like this:

Modify GH workflow

Sweet. Now, we only have to tell our GH Actions workflow to run that script. The easiest way is to add another step after all dependencies are installed.

Add AWS Credentials to GitHub Secrets

But if you think about it, this cannot work yet. The machine that is supposed to execute this code, doesn’t know our GitHub credentials. If this script works on the GH action, then you should seriously check if you’ve accidentally pushed your .Rprofile file (which you should absolutely not do.)

So that’s why we have to tell our step about the environment variables it needs.

And then in your GitHub repository, go to Settings > Secrets and variables > Actions, and add:

• AWS_ACCESS_KEY_ID

• AWS_SECRET_ACCESS_KEY

• AWS_DEFAULT_REGION

And that’s it! Now every time the flow runs, we will have added a new row to our dataset.

You now have a working automation that pulls data from anywhere you like and stores it in cloud storage securely and automatically.

As always, if you have any questions, or just want to reach out, feel free to contact me by replying to this mail or finding me on LinkedIn or on Bluesky.

See you next week,
Albert 👋

Enjoyed this newsletter? Here are other ways I can help you:

Data Cleaning With R Master Class

A data scientist’s guide to avoid wasting time with data cleaning. Learn 5 fundamental aspects of data cleaning, get to your insights much quicker.

Insightful Data Visualizations for "Uncreative" R Users

In this course, I teach you how to create insightful data visualizations without being a design expert.

Making Beautiful Tables with R

Tables are often boring, hard to read, and, most importantly, they don't communicate effectively. In this course, I will show you that creating beautiful tables is just a matter of learning a few important principles.

• My playlist on YouTube

• Free book: Yet Again - R + Data Science

• Free book: Beautiful tables with {gt}

• Blog posts

Guten Tag!

Many greetings from Munich, Germany. In this week’s newsletter, I’ll show you how to use the {pins} package to store your data on different online platforms. Basically, this is a follow-up to last week’s newsletter.

And there are many places where you can store pins. Here, we’ll walk through two particuclar examples. Namely, GitHub and AWS S3. But before I show you how they work, let me show you my usual announcements.

Interactive Documents > Full App

Ever found yourself building a Shiny app that contains mostly text with just a few interactive parts? That’s a bit of a nuisance to set up because you’re mostly trying to make your text look nice inside your app’s big blank canvas.

Now here’s a better way.

Just create a Quarto document! This will immediately use formatting that is proper for text-heavy documents.

And the interactive part?

Well, in Quarto you can add them via Shiny server code chunks. And in my newest YT video I show you how that works. Check it out:

Using GitHub as a Pin Board

Let’s assume you have a public GitHub repository with pins stored in it. You can connect to it using the board_url() function. Here’s an example of that using pins from the public {pins} repo.

Once you’ve created the board, you can read pins from it using the pin_read() function:

But this only works with pins that contain actual R data sets. For example the raw pin contains a .txt file. That’s why it can’t be opened in the R console.

Instead, you have to download the file.

Using GitHub with Private Repos

What if you want to use a private repository instead?

In that case, you’ll need to include your GitHub Personal Access Token in the request. You do this by first creating a named vector with your token as the authorization header.

And then you could pass it to the headers argument of board_url(). Here, I’ve just taken the example from the {pins} docs.

Limitations of Pins with GitHub

Note that using pins like that only allows reading from GitHub. It doesn’t let you modify the data. If you want to change pins from a GitHub repo, you’d have to:

1. Clone the repository

2. Modify the pins locally

3. Push the changes back to GitHub

This isn’t the most convenient workflow. That’s why I prefer other options like AWS S3.

Using AWS S3 as a Pin Board

For a smoother experience, AWS S3 is a great option.
To use it, you need an existing S3 bucket to store your pins. If you don’t have one yet, check out the tutorial I wrote a few weeks back on how to create an S3 bucket.

Once you have your bucket and region ready, you can connect using the board_s3() function:

If you encounter an “Access Denied” error, it’s likely because your AWS credentials are not set up correctly. You need to have the following environment variables configured:

• AWS_ACCESS_KEY_ID

• AWS_SECRET_ACCESS_KEY

You can set these in your .Renviron file or in your R session.

Writing Pins to AWS S3

Once your S3 board is set up and authenticated, you can read and write pins just like with a local board.

The key benefit here is that you don’t need to manually download or push data. {pins} handles it all automatically. In fact, if you take a look at your S3 bucket now, you’ll see that the data is already there.

With that, you have two solid ways to store your pins online. Next week, I’ll show you how to set up an automation on GitHub Actions that automatically collects time series data and stores it using the {pins} package.

As always, if you have any questions, or just want to reach out, feel free to contact me by replying to this mail or finding me on LinkedIn or on Bluesky.

See you next week,
Albert 👋

Enjoyed this newsletter? Here are other ways I can help you:

Data Cleaning With R Master Class

A data scientist’s guide to avoid wasting time with data cleaning. Learn 5 fundamental aspects of data cleaning, get to your insights much quicker.

data-cleaning.albert-rapp.de

Insightful Data Visualizations for "Uncreative" R Users

In this course, I teach you how to create insightful data visualizations without being a design expert.

arapp.thinkific.com/courses/insightful-data-visualizations-for-uncreative-r-users

Making Beautiful Tables with R

Tables are often boring, hard to read, and, most importantly, they don't communicate effectively. In this course, I will show you that creating beautiful tables is just a matter of learning a few important principles.

rfortherestofus.com/courses/tables

• My playlist on YouTube

• Free book: Yet Again - R + Data Science

• Free book: Beautiful tables with {gt}

• Blog posts

Guten Tag!

Many greetings from Munich, Germany. In today’s newsletter, we are talking about the {pins} package. This package is not very well known in the R sphere, but it is a fantastic tool for keeping track of different data versions.

Basically, {pins} makes storing data and versioning dead-simple. You can think of it as Git for your data. But before we dive in, I have a short thing to ask you: ‘

What‘s next?

Recently, I find myself covering more and more technical aspects of data science that are not necessarily R focused. This includes:

• Building automations with GitHub Actions

• Setting up infrastructure on AWS or Azure

• Using JavaScript to handle AI responses in Shiny apps

I do this because I think data science doesn’t happen in any single tool or language. Thus, I want to continue writing in as broad a scope as I feel like at the moment (after all I get to do whatever I want on this newsletter 🤪)

But I do want to see if I can make this as valuable for you as possible. So let me know what you’re interested in by filling out this small poll.

Just click on your favorite choice and it will be recorded. After that, you’ll also have the chance to leave additional comments in case you want to suggest something I haven’t thought of.

Why use the {pins} package?

Now, before I show you how {pins} works, let’s talk about why you should even use it

• Version control: Prevents accidental overwriting of important datasets when re-running workflows.

• Efficient workflows: You can run a calculation or an API call once and store the data permanently in a pin.

• Metadata storage: {pins} keeps detailed information about your data so you can easily restore or revert to previous versions.

• Seamless integration : You can store data anywhere you like. GitHub, Google Drive, AWS S3, you name it. {pins} will work exactly the same.

Surely, you’re wondering how all of that is supposed to work. So let’s dive into the technical details.

Set up {pins}

First, you need to install the package. As always, you can do this with:

Set up a board

To store data, you need to place “pins” in a “board”. If you want to try {pins} locally first, create a board using:

Write data to a pin

To write data to your board, simply use:

This creates a new pin, and the best part is that previous data versions aren’t deleted.

Retrieving a specific version

To check how many data versions exist in your board, use:

That’s currently pretty boring. So let’s modify our data and overwrite our pin multiple times.

Now, check out the different versions:

Once you decide which version you want to load, retrieve it using:

As you can see, setting up {pins} is simple and efficient for storing and retrieving data. And the best thing is that you cannot accidentally overwrite data. Next week, I’ll show you how to set up cloud storages for your pins.

As always, if you have any questions, or just want to reach out, feel free to contact me by replying to this mail or finding me on LinkedIn or on Bluesky.

See you next week,
Albert 👋

Enjoyed this newsletter? Here are other ways I can help you:

Data Cleaning With R Master Class

A data scientist’s guide to avoid wasting time with data cleaning. Learn 5 fundamental aspects of data cleaning, get to your insights much quicker.

Insightful Data Visualizations for "Uncreative" R Users

In this course, I teach you how to create insightful data visualizations without being a design expert.

Making Beautiful Tables with R

Tables are often boring, hard to read, and, most importantly, they don't communicate effectively. In this course, I will show you that creating beautiful tables is just a matter of learning a few important principles.

• My playlist on YouTube

• Free book: Yet Again - R + Data Science

• Free book: Beautiful tables with {gt}

• Blog posts

Guten Tag!

Many greetings from Munich, Germany. In today’s newsletter we change the authentication for our Quarto project so that we can decide which users get to access our content. This will be similar to out GitHub authentication from last week.

But this time, we can restrict access to only a few selected persons (if we want that). And to do so, we’re going to use another service from Azure called “Azure Active Directory B2C”.

Create a new tenant

The first thing that you have to do is to create a new tenant for Azure AD B2C. To do so, you create a new resource.

And then, you select Azure AD B2C:

There you will have to specify the org name and domain name of your tenant. Just choose what you like (as long as Azure accepts the name.)

Switch tenant

Once you’ve created the tenant, you can switch to it via the settings icon at the top of the Azure portal.

Just remember that your Azure Static Web App now lives in a separate tenant. That’s why you’ll have to switch between tenants when you modify the app settings.

Register an application

In our new tenant, we can access the Azure AD B2C resource. There, we first have to register an application. This is like registering the OAuth app that we created last week on GitHub.

In the settings of that registration, you will have to enable all accounts to be allowed to “authenticate via user flows”. Also, make sure that the redirect URI of that application points to our app.

Watch out: The redirect URI now doesn’t contain .auth/login/github but instead contains .auth/login/aadb2c.

Client ID & Secret

Now that we have an OAuth app, we need to get client ID and client secret. This is exactly like what we did last week. The client ID is immediately displayed in the app page and the secret can be generated via the “Certificates & secrets” page.

There, you can generate a new secret.

Just make sure that you copy the secret as soon as you’ve created it. Otherwise, it will be lost as the secret will only be displayed once.

Create a new user flow

Remember how we had to enable “user flows”? You can think of this as the procedures that a user might want to go through. Common procedures are “sign in”, “sign up”, “reset password” or “edit profile”.

And that’s also what we need to set up inside of our AD B2C resource. We can do so via the “Policies” page.

There, we can create a new sign in flow so that users can sign in.

The rest is pretty straightforward: Pick a name and choose sign up methods.

If you want, you can also turn off multi-factor authentication and determine what information are transmitted when a user signs in.

Once you’re finished, you can see a new user flow in your Policies page. Notice that the name of the policy also includes “B2C_1_”. This little detail will become important in a sec.

Modify config file

Now that we have everything set up, we can change the auth settings of our web app. Once again, this only requires modifying the staticwebapp.config.json file. Luckily for us, the Azure docs also have the perfect auth code snippet for us:

Here, we need to replace

• <TENANT_NAME> with the domain name of our tenant, and

• <POLICY_NAME> with the user flow name that we have created earlier. Remember to use the name that includes “B2C_1_”.

Also, we can modify the requested scopes a bit. All in all, this should look something like this:

Finally, you should make sure that your redirect now moves users to .auth/login/aadb2c. Overall, the config file should look like this.

Once you’ve made these changes, make sure to rebuild your Quarto project and push the changes to our GH repo.

Set environment variables

Finally, the remaining thing that we have to do is set up environment variables inside of our app. You see, inside the auth snippet we now use AADB2C_PROVIDER_CLIENT_ID and AADB2C_PROVIDER_CLIENT_SECRET. These need to be set inside the environment variables page of our Azure Static Web App.

Redirect to login

Nice! We’ve set everything up so it’s time to test it. When you try to access the website of your Quarto project, you should be redirected to the following login screen.

While this is nice, it doesn’t matter what you put in there. The system won’t be able to find your user details.

Clearly, this happens because we’ve never defined a user base.

Add a user

We can add a user to our ADB2C resource via the Users page.

There, we just have to set up a new user.

The page just requires defining an e-mail address and a password. Once we have set that up we can use this mail and the password to sign up via our login page. This will successfully authenticate us and redirect us to our website.

And with that we have successfully changed the authentication once again 🥳 But this time we can customize who gets to sign in and who doesn’t.

As always, if you have any questions, or just want to reach out, feel free to contact me by replying to this mail or finding me on LinkedIn or on Bluesky.

See you next week,
Albert 👋

Enjoyed this newsletter? Here are other ways I can help you:

Data Cleaning With R Master Class

A data scientist’s guide to avoid wasting time with data cleaning. Learn 5 fundamental aspects of data cleaning, get to your insights much quicker.

Insightful Data Visualizations for "Uncreative" R Users

In this course, I teach you how to create insightful data visualizations without being a design expert.

Making Beautiful Tables with R

Tables are often boring, hard to read, and, most importantly, they don't communicate effectively. In this course, I will show you that creating beautiful tables is just a matter of learning a few important principles.

• My playlist on YouTube

• Free book: Yet Again - R + Data Science

• Free book: Beautiful tables with {gt}

• Blog posts

Guten Tag!

Many greetings from Munich, Germany. Last week, we enabled authentication for our Quarto website that is hosted on Azure Static Web Apps. But this authentication allowed everybody with a Microsoft account to sign in. That’s usually not what we want. So let me show you how to modify the authentication settings in today’s newsletter.

Recap

If you recall, all we had to do to add authentication was to include a file called staticwebapp.config.json. Here’s how it looked:

This is the configuration that redirected unauthenticated users to https://website.com/.auth/login/aad. That’s where users could then sign in with their Microsoft account. And to be more precise, users signed in with their Azure Active Directory (AAD) account. This name still floats around everywhere even though that service is called Azure EntraID nowadays.

Add custom authentication

And the nice thing is that we can change the authentication via the same config file. All we have to do is check out this snippet I stole from the Azure documentation.

This snippet allows users to authenticate with their GitHub account. This stil doesn’t restrict the user base that can access our website by a lot. But it’s the first step to understanding how to customize authentication.

In this snippet, we have two placeholders that we should acknowledge: GITHUB_CLIENT_ID and GITHUB_CLIENT_SECRET_APP_SETTING_NAME.

Actually, there’s also a third name that we should notice: In the snippet you can see that we use "github" as an identity provider name. This means that when we throw this snippet into staticwebapp.config.json, then we should also change the redirect URL.

Failed redirect

Now, if we push these changes to our GitHub repository and try to access our website, we get this error.

Very sad, I know. But it’s also kind of expected. After all, we didn’t do anything with the names GITHUB_CLIENT_ID and GITHUB_CLIENT_SECRET_APP_SETTING_NAME. So let’s change that.

Create a GitHub Client

As we can infer from the names GITHUB_CLIENT_ID and GITHUB_CLIENT_SECRET_APP_SETTING_NAME, we need a so-called client. And that is typical for any identity provider. Whether that’s GitHub or Azure Active Directory (AAD), you need a client to handle the back and forth of credentials.

And the only reason why we didn’t have to do that so far is because the default AAD setting in Azure Static Web Apps uses a default client. But the moment we move away from that, we also have to provide our own client. Luckily, it’s usually quite easy to create one after you found the right menus to click. Let’s see how to do that with GitHub.

In your GitHub account settings, you can find the developer settings at the bottom of the sidebar.

In there, you can create a new OAuth app.

These kind of apps implement the so-called OAuth flow that is commonly used for authentication. Here, I have already created such an app with the name “QuartoAuthenticationTest”.

But let’s create a new one here. That way, I can show you all the necessary settings.

As you can see, you need to register the URL of your hosted website with the client. More importantly, you also need to declare a “callback” url. This is the URL where your authentication service expects credentials to arrive. As per the Azure docs, that URL is always just the regular URL followed by .auth/login/<IDENTITY_PROVIDER>/callback.

Get client id and secret

Once you created the OAuth app, you can see the “client ID” on the page that you are being redirected to. Also, you can create a “client secret”.

Fill environment variables

Equipped with ID & secret you can now fill the corresponding environment variables. This doesn’t mean that you have to modify the config file. Instead you have to login to Azure Static Web Apps and set environment variables for your app.

There, you will need to create the variables GITHUB_CLIENT_ID and GITHUB_CLIENT_SECRET_APP_SETTING_NAME. After all, these are the variable names you have specified in your config file.

And once all settings were saved, the redirect to GitHub should work properly. That way, you will see the following GitHub page when you want to access your Quarto website.

And once you sign in, you will be redirected to your Quarto page. Hooray.

Nice! We successfully changed the authentication from AAD to GitHub. That’s not very restrictive but the good news is that the principles with Azure Active Directory are pretty much the same. Thus, you can restrict access to your desired user base. I’ll show you how that works next week.

As always, if you have any questions, or just want to reach out, feel free to contact me by replying to this mail or finding me on LinkedIn or on Bluesky.

See you next week,
Albert 👋

Enjoyed this newsletter? Here are other ways I can help you:

Data Cleaning With R Master Class

A data scientist’s guide to avoid wasting time with data cleaning. Learn 5 fundamental aspects of data cleaning, get to your insights much quicker.

Insightful Data Visualizations for "Uncreative" R Users

In this course, I teach you how to create insightful data visualizations without being a design expert.

Making Beautiful Tables with R

Tables are often boring, hard to read, and, most importantly, they don't communicate effectively. In this course, I will show you that creating beautiful tables is just a matter of learning a few important principles.

• My playlist on YouTube

• Free book: Yet Again - R + Data Science

• Free book: Beautiful tables with {gt}

• Blog posts

Guten Tag!

Many greetings from Munich, Germany. In today’s newsletter, we want to add authentication to the Quarto project we deployed last week. If you recall, we took a simple Quarto book/website and deployed it via Azure Static Web Apps. And with that set up, it’s actually super easy to add a layer of authentication in front of that.

But before I show you how that works, it’s time for my regular announcements.

DCMC 4 finished

This week, I uploaded the final lesson of my data cleaning master class. It ties together all the things we have learned throughout the lessons of Part 4. What’s more is that it applies the lessons to a real-world scenario where we run an analysis on time series data.

If you’re interested in leveling up your skills regarding time-related data, then you can join 100+ learners via

Data Cleaning With R Master Class

This course teaches you everything you need to know about cleaning data in R fast & efficiently.

And as a little celebration that Part 4 is complete now, you can use the code “DCMC4” to get 10% off. Also, this code is only valid until March 2.

PDFs to DataFrame

Also, this week I released a new video that shows viewers how to use the {ellmer} package to

• extract specific informations from texts & PDFs and

• bring that into a DataFrame format.

You can check out the video at

One config file to rule them all

Now, let’s get back to adding authentication to a Quarto website. It’s actually not as hard as you’d think. Everything is governed by one single JSON file called staticwebapp.config.json.

Basically, what you need to do is fill this JSON file with specific keywords and values. That way, your website will change all kinds of settings. And that’s how you can easily add authentication.

And the good news is that you don’t have to know all the keywords yourself. Inside the Azure documentation, there are lots of config snippets for all kinds of use cases. For example, there’s a config snippet to “restrict access to the entire application.”

Sounds exactly like the thing we want. Here’s how it looks.

Understanding the routes

Now, this might feel intimidating. But there’s actually a very simple logic to the snippet we’ve just seen.

You see, via the routes setting you can define specific rules for how website visitors should be routed. Here, the routes settings are only filled with one setting, namely this:

Here, the route is defined by / followed by anything (as denoted by the asterisk.) This means that users wanting to access, say, https://website.com/subsite, will be targeted by this rule. You know, because our URL root https://website.com is followed by /*.

And via the allowedRoles setting inside the routing rule, we can specify which kind of roles a user must have to access the desired page. Here, only users with the role “authenticated” are allowed. This role is already build into any Azure Static Web App.

Can I add other routes?

To make this example a little more clear, let’s play around with the routes. We could say that the start page index.html is available to anyone. But all other sites are restricted to authenticated users.

In that case we need to add more rules to the routes settings. Here’s how that could look:

In our Quarto book setting this would mean user can access the start page but no other. But due to the fact that Quarto saves all the style files of any website inside a separate directory and not directly in a file like index.html, the page will look ugly to the unauthenticated users.

This happens because the user then doesn’t have permissions to access the style files. You can avoid that kind of behavior by making only the index.html file self-contained by adding embed-resources: true inside the YAML header of index.qmd. But for now, let’s just assume that we want to restrict all pages.

What happens to unauthenticated users?

This begs the question “What happens to unauthenticated users?”. Well, the simple answer is that they get denied access. The more specific answer is that the server behind our Azure static web app will return a “401 Unauthorized” response.

Here, 401 is the standard status code that all web applications know and use. But we don’t want the user’s browser to just get this message. Instead, we want to tell it to maybe first authenticate somewhere else and come back after that.

This means that instead of serving the user with a “401 Unauthorized”, we want to serve the user’s browser with a “302 Moved Temporarily” status. Additionally, we want to tell the browser where things were moved to. That way, the browser knows where to redirect its request to.

Here, the redirect will lead to https://website.com/.auth/login/aad. This is the standard authentication with Azure Active directory (nowadays called EntraID). For now, you don’t need to know this service.

All you need to know is that it’s a service that can store user credentials. Also, the URL the user is then redirected to will prompt the user to authenticate with this service.

Integrate config into Quarto

So now that we know how to read the config file, let’s talk about how to integrate it into your Quarto project. You see, Azure static web apps currently only looks at the rendered _book directory in your repo on GitHub.

But your config file will not be automatically copied to this directory when the project is rendered. Hence, you need to do two things.

1. Create the staticwebapp.config.json in the root directory of your project

2. Modify your _quarto.yml file as follows:

   

This will instruct Quarto to copy the config file into the _book directory when the project is rendered. Now, all that’s left to do is to re-render your project and push all the changes to GitHub.

Did it work?

This begs the question: “Is our project secure now?” Well, let’s check. First, we make sure that the corresponding GH Action ran successfully:

Then, we can head to the URL of our hosted project. This is how we’re greeted now:

You may have seen such a window at your workplace before. Congrats, you now understand that the service behind this is Azure Active Directory (EntraID).

Now you’ll have to login with a Microsoft account. Once you click the button, you’ll see that it asks you to grant permissions.

And once you grant the permissions, you are redirected to your website. This time you can see your Quarto project, though.

And in case you’re wondering: You really are authenticated at this point. You can check this by heading to https://website.com/.auth/me. This page now contains a JSON with your user credentials.

Hooray! We have a first layer of authentication now. But it doesn’t stop anyone with a Microsoft account to access our page. After all, we never specified a specific user pool that can authenticate successfully. Let’s talk about this next week.

As always, if you have any questions, or just want to reach out, feel free to contact me by replying to this mail or finding me on LinkedIn or on Bluesky.

See you next week,
Albert 👋

Enjoyed this newsletter? Here are other ways I can help you:

Data Cleaning With R Master Class

A data scientist’s guide to avoid wasting time with data cleaning. Learn 5 fundamental aspects of data cleaning, get to your insights much quicker.

Insightful Data Visualizations for "Uncreative" R Users

In this course, I teach you how to create insightful data visualizations without being a design expert.

Making Beautiful Tables with R

Tables are often boring, hard to read, and, most importantly, they don't communicate effectively. In this course, I will show you that creating beautiful tables is just a matter of learning a few important principles.

• My playlist on YouTube

• Free book: Yet Again - R + Data Science

• Free book: Beautiful tables with {gt}

• Blog posts

Guten Tag!

Many greetings from Munich, Germany. Today, I’ll show you how to host a Quarto project on Azure. You know, the huge cloud provider from Microsoft.

And I know that this sounds a bit like overkill when you can easily publish Quarto docs somewhere else. But this will actually set the stage for adding our own layer of authentication later on.

You see, Azure has a few nice features that make it possible to host your Quarto project with authentication. That way, you could deploy Quarto projects and make it accessible only to a targeted audience.

But authentication is what we’ll cover next week. First, we have to get our project to Azure. But before we dive in, time for my usual announcements.

New Time Data Cleaning Tips

This week, I’ve uploaded two new videos for Part 4 of my Data Cleaning Master Class. These lessons will show you how to

• calculate moving averages for time series and

• how to check whether your time series data is complete.

These videos also conclude the bite-sized lessons part of Part 4 which means that there’s only the real-world example left. In this real-world example, we’re actually looking at my own data from my content creation career. That way, students get to apply techniques that I use myself to check on my YouTube and LinkedIn stats.

I’ll upload that 28 minute deep-dive next week. So, if you want to join 100+ students, you can do so at

Data Cleaning With R Master Class

This course teaches you everything you need to know about cleaning data in R fast & efficiently.

5 Levels of Data Cleaning

I’ve also released a new YT tutorial this week. It shows you the same data cleaning project in 5 different ways. And at every stage the code will get shorter and more efficient. That way, you’ll learn to write less code and get your results much quicker:

Let’s create a dummy project

In order to deploy anything, we first need a new Quarto project. You can do so via the RStudio “New Project” dialogue.

Let’s make it easy and create a new Quarto book.

But make sure that you also use a git repo.

Once you’ve set that up, make sure that you have rendered your project. That way, you should see something like this:

Create a GitHub repo

Next, the code for our Quarto project needs to be hosted on GitHub. So make sure that you create a new repo on GitHub.

And once you have that, you can push your code to the repository. This should look something like this:

Create a Static Web App on Azure

Great! Now, we just have to login to Azure (or create an account if you don’t have one yet.) There, we want to jump to the Static Web Apps Service. This service makes it easy to deploy static web pages (like our Quarto project.)

At first you will be prompted to assign a resource group. This is nothing but an administrative unit to keep an eye on costs for billing purposes. Hence, you can create a new resource group for this project. Here I’ve just called it Quarto_test.

Then, our app will need a name. I went with “3mw-demo” but feel free to use a name that suits you.

Choose a plan

Next, we need to choose a plan.

If you don’t need authentication, you can actually deploy your Quarto project for free. But since we want to add authentication, we will need to go for the paid tier that currently costs $9 per month.

However, if you just want to try it out, you can just delete the static web app after you’re done and you don’t have to pay the full month. In that case, Azure will bill you only for the hours you’ve used.

Linking your GitHub repo

Finally, you need to link to your GitHub repo and the specific branch you want to deploy from.

And once that is done, Azure will ask you what kind of app you want to deploy. Since we don’t use any of the common web frameworks like Vue, Svelte, etc. we just pick “Custom” (if Azure doesn’t detect that on its own.) Then, we only have to tell Azure that all the necessary files are in the _book directory. So just set the “App location” to ./_book.

Deploy the Quarto project

And that’s it. Now you can hit the “review + create” button. After you’ve confirmed the details, Azure will deploy your app.

This will mean two things:

First, you’ll see a new GitHub Actions workflow in your repository. This means that whenever you push changes to your repo, your deployed app will be updated via this workflow.

Second, on the main page of the Static Web Apps service you can now see your new app. If you click into it, you’ll see lots of details of your web app.

And if you head to the mentioned URL you’ll see that your Quarto project was deployed. Have a look.

Hooray! With that we have successfully deployed Quarto on Azure. Next week, we’ll add authentication to that. As always, if you have any questions, or just want to reach out, feel free to contact me by replying to this mail or finding me on LinkedIn or on Bluesky.

See you next week,
Albert 👋

Enjoyed this newsletter? Here are other ways I can help you:

Data Cleaning With R Master Class

A data scientist’s guide to avoid wasting time with data cleaning. Learn 5 fundamental aspects of data cleaning, get to your insights much quicker.

Insightful Data Visualizations for "Uncreative" R Users

In this course, I teach you how to create insightful data visualizations without being a design expert.

Making Beautiful Tables with R

Tables are often boring, hard to read, and, most importantly, they don't communicate effectively. In this course, I will show you that creating beautiful tables is just a matter of learning a few important principles.

• My playlist on YouTube

• Free book: Yet Again - R + Data Science

• Free book: Beautiful tables with {gt}

• Blog posts

Guten Tag!

Many greetings from Munich, Germany. In today’s newsletter we’re using OCR services from AWS. This lets us extract texts from any kind of document or image. That way, we can extract information from all sorts of documents and feed that into text processing tools like LLMs.

But before we do that, it’s time for my regular announcements:

New Lessons On Data Cleaning With Time Data

Like every week, my data cleaning master class is making a lot of progress. This week, I’ve uploaded 6 new bite-sized lessons:

• Lesson 07: Time length calculations with durations & periods

• Lesson 08: Calculate time between to time points

• Lesson 09: Introducing intervals

• Lesson 10: Using interval functions

• Lesson 11: Rounding dates

• Lesson 12: Filter time columns

All of these small little things add up so that you can easily handle any time-related data that comes your way. If you’re interested in joining 100+ learners, you can do so via the course landing page:

Data Cleaning With R Master Class

This course teaches you everything you need to know about cleaning data in R fast & efficiently.

Using AI Functions

If you’ve enjoyed the newsletter from a few weeks ago, then I’m sure you’re delighted to hear that there is also a video version available now. This tutorial shows you how you can use AI tools with {ellmer}.

Get a Textract client

Just like last time, we need to get a client for the specific AWS service we want to use. Here, that’s Textract.

And with this client’s start_document_text_detection() method we can send an object that we have stored in S3 to the text detection service. Here, we will just use the PDF we have stored in S3 last time.

Handle the response

Notice that we saved the result of the start processing call in a variable called response. Let’s check out what’s in it.

This response can be used together with the get_document_text_detection() method.

Make sense of the output

Woah! That’s a lot of stuff. Thankfully, the {paws} documentation offers some clarity.

With that we know that there is a field called JobStatus. This one can be either "IN_PROGRESS", "SUCCEEDED", "FAILED" or "PARTIAL_SUCCESS".

So, when the OCR job is still in progress, there’s nothing to do but to wait and check in every few seconds.

Check for next tokens

Sweet. Our text processing job finished. Let’s first check the NextToken field in the results.

This one is empty. This means that all OCR’ed texts fit into a single response. That’s great so we don´t have to loop through all the NextTokens.

You see, for long documents it can happen that you will have to first collect all responses in a list. The code for this could look something like this:

Thankfully, we don’t have to do that here. Thus, we can directly dive into the blocks of text we want to extract.

Extract Blocks

First, we need to find all the blocks that are of type “LINE”.

From that we can extract the texts:

Send to AI

Nice! We have extracted all the texts. With that information we can let AI handle the rest. Let’s first recall how our document looks:

Now, we define a chat and an invoice position type as discussed a few weeks ago.

And then we ask our AI to extract the information:

Hoooray! This looks like exactly the content we need. As always, if you have any questions, or just want to reach out, feel free to contact me by replying to this mail or finding me on LinkedIn or on Bluesky.

See you next week,
Albert 👋

Enjoyed this newsletter? Here are other ways I can help you:

Data Cleaning With R Master Class

A data scientist’s guide to avoid wasting time with data cleaning. Learn 5 fundamental aspects of data cleaning, get to your insights much quicker.

Insightful Data Visualizations for "Uncreative" R Users

In this course, I teach you how to create insightful data visualizations without being a design expert.

Making Beautiful Tables with R

Tables are often boring, hard to read, and, most importantly, they don't communicate effectively. In this course, I will show you that creating beautiful tables is just a matter of learning a few important principles.

• My playlist on YouTube

• Free book: Yet Again - R + Data Science

• Free book: Beautiful tables with {gt}

• Blog posts