Building Multilingual Data Science Teams

Language Wars!

The Reality

Amost everything that used to make Python awesome that wasn’t in R has been since ported over to R.

And everything that used to make R awesome that wasn’t in Python has been since ported over to Python.

Why Have a Multilingual Team?

Larger potential talent pool
More tools in the toolbox
Dual offering

The Key to a Great Multilingual Data Science Team?

How Empathy Manifests Technically

🌍 Environment management

📦 Package choices

📝 Documentation

How Empathy Manifests Technically

Environment Management

“But it works on my machine”

Environment Management

Operating System
Python and/or R version
Package versions

Environment Management

Operating System & Python / R Version

Third-Party Tools

Posit Workbench

Open Source tools

Containers (Docker / Podman)
Nix

Package Version

Python

uv, venv, pipenv, etc.

R

{renv}
{packrat}

On the Operating System & Python/R Version side, you can typically use the same tooling for managing both
There are two best approaches to doing this:
- Leveraging third party tools like Posit Workbench hosted on a dedicated server or cloud that allow you to set up your environment in more simplified, point-and-click approach
- Or leveraging open source tooling where you will essentially script your environment
Going the open source approach requires some DevOps skillsets
- So you’ll need to ask yourself if that’s something you want to undertake or not, and weigh that against the cost / benefit of something like a Posit Workbench that abstracts a lot of that away for you
On the package versioning side, using virtual environments – which allow you to manage package versions on a project-by-project basis, instead of having some sort of global library for all projects – can be really efficient for collaboration and ensuring you and the rest of your team are working within the same setup
- I think there are essentially two leaders these days: uv on the Python side, and {renv} on the R side

Package Choices

Package Choices: dplyr & polars

library(dplyr)

df <-
    mtcars |>
        filter(
            mpg > 20
        ) |>
        select(wt) |>
        arrange(desc(wt)) |>
        head(5)
#

import polars as pl

df = (
    mtcars
    .filter(
        (pl.col("mpg") > 20)
    )
    .select("wt")
    .sort("wt", descending=True)
    .head(5)
)

In these next few slides, you’ll see R code examples on the left side, and Python code examples on the right side of the screen
This goes somewhat back to my slide where I mentioned that everything great in each language had been ported to the other language, but it somewhat goes beyond that; I think the great ideas are being borrowed from each language
The python package {polars} is a great example. The author of polars, Ritchie Vink, has mentioned in a bunch of interviews that {polars} takes a lot inspiration from the tidyverse in R, and in a minute here I think we’ll show you just how true that is
We know that data prep is 80% of data science, so the methodology and opinions you choose for your data prep code is important
At Ketchbrook, our choice is typically {polars} for Python data prep or {dplyr} for R data prep, because, not only are they two of the most popular data wrangling libraries in their respective languages, but they are also syntactically very similar

Package Choices: dplyr & polars

library(dplyr)

df <-
    mtcars |>
        filter(
            mpg > 20
        ) |>
        select(wt) |>
        arrange(desc(wt)) |>
        head(5)
#

import polars as pl

df = (
    mtcars
    .filter(
        (pl.col("mpg") > 20)
    )
    .select("wt")
    .sort("wt", descending=True)
    .head(5)
)

Package Choices: dplyr & polars

library(dplyr)

df <-
    mtcars |>
        filter(
            mpg > 20
        ) |>
        select(wt) |>
        arrange(desc(wt)) |>
        head(5)
#

import polars as pl

df = (
    mtcars
    .filter(
        (pl.col("mpg") > 20)
    )
    .select("wt")
    .sort("wt", descending=True)
    .head(5)
)

Package Choices: dplyr & polars

library(dplyr)

df <-
    mtcars |>
        filter(
            mpg > 20
        ) |>
        select(wt) |>
        arrange(desc(wt)) |>
        head(5)
#

import polars as pl

df = (
    mtcars
    .filter(
        (pl.col("mpg") > 20)
    )
    .select("wt")
    .sort("wt", descending=True)
    .head(5)
)

Package Choices: dplyr & polars

library(dplyr)

df <-
    mtcars |>
        filter(
            mpg > 20
        ) |>
        select(wt) |>
        arrange(desc(wt)) |>
        head(5)
#

import polars as pl

df = (
    mtcars
    .filter(
        (pl.col("mpg") > 20)
    )
    .select("wt")
    .sort("wt", descending=True)
    .head(5)
)

Package Choices: DuckDB

library(duckdb)

# Connect to DuckDB (in-memory db)
conn <- dbConnect(duckdb::duckdb())

# Define SQL query
query <- "
    SELECT wt
    FROM mtcars
    WHERE mpg > 20
    ORDER BY wt DESC
    LIMIT 5
"

# Execute query and get results
result <- dbGetQuery(conn, query)

import duckdb

# Connect to DuckDB (in-memory db)
conn = duckdb.connect()

# Define SQL query
query = """
    SELECT wt
    FROM mtcars
    WHERE mpg > 20
    ORDER BY wt DESC
    LIMIT 5
"""

# Execute query and get results
result = conn.execute(query).df()

Package Choices: DuckDB

library(duckdb)

# Connect to DuckDB (in-memory db)
conn <- dbConnect(duckdb::duckdb())

# Define SQL query
query <- "
    SELECT wt
    FROM mtcars
    WHERE mpg > 20
    ORDER BY wt DESC
    LIMIT 5
"

# Execute query and get results
result <- dbGetQuery(conn, query)

import duckdb

# Connect to DuckDB (in-memory db)
conn = duckdb.connect()

# Define SQL query
query = """
    SELECT wt
    FROM mtcars
    WHERE mpg > 20
    ORDER BY wt DESC
    LIMIT 5
"""

# Execute query and get results
result = conn.execute(query).df()

Package Choices: DuckDB

library(duckdb)

# Connect to DuckDB (in-memory db)
conn <- dbConnect(duckdb::duckdb())

# Define SQL query
query <- "
    SELECT wt
    FROM mtcars
    WHERE mpg > 20
    ORDER BY wt DESC
    LIMIT 5
"

# Execute query and get results
result <- dbGetQuery(conn, query)

import duckdb

# Connect to DuckDB (in-memory db)
conn = duckdb.connect()

# Define SQL query
query = """
    SELECT wt
    FROM mtcars
    WHERE mpg > 20
    ORDER BY wt DESC
    LIMIT 5
"""

# Execute query and get results
result = conn.execute(query).df()

Package Choices: ggplot2 & plotnine

library(ggplot2)


ggplot(
  mtcars,
  aes(x = wt, y = mpg)
) +
  geom_point(color = "red") +
  geom_smooth(method = "lm")
#

from plotnine import

(
  ggplot(
    mtcars,
    aes(x = "wt", y = "mpg")
  ) +
    geom_point(color = "red") +
    geom_smooth(method = "lm")
)

Package Choices: gt & great-tables

library(gt)

int_cols <- c(
  "cyl", "vs", "am",
  "gear", "carb"
)


gt(mtcars) |>
  tab_header(
    title = "Awesome mtcars",
    subtitle = "With 💙 + GT"
  ) |>
  fmt_number() |>
  fmt_integer(columns = int_cols)
#

from great_tables import GT

int_cols = [
  "cyl", "vs", "am",
  "gear", "carb"
]

(
  GT(mtcars)
  .tab_header(
      title = "Awesome mtcars",
      subtitle = "With 💙 + GT"
  .) |>
  .fmt_number()
  .fmt_integer(columns=int_cols)
)

Documentation

Documentation: Function Documentation Syntax

round_up <- function(x, dig) {
    f <- 10 ** dig
    out <- ceiling(x * f) / f
    return(out)
}

def round_up(x, dig):
    f = 10 ** dig
    out = np.ceil(x * f) / f
    return out

Documentation: Function Documentation Syntax

#' Round a number *up* to a
#' certain number of digits.
#'
#' @param x (double) The value
#'   to be rounded.
#' @param dig (int) The
#'   number of digits to round
#'   to.
#'
#' @return The rounded number.
#'
#' @examples
#' # This returns `2.15`
#' round_up(2.141, 2)
round_up <- function(x, dig) {
    f <- 10 ** dig
    out <- ceiling(x * f) / f
    return(out)
}

def round_up(x, dig):
    """
    Round a number *up* to a
    certain number of digits.

    Parameters
    ----------
    x : float
        The value to be
        rounded.
    dig : int
        The number of digits to
        round to.

    Returns
    -------
    float
        The rounded number.

    Examples
    --------
    >>> round_up(2.141, 2)
    2.15
    """
    f = 10 ** dig
    out = np.ceil(x * f) / f
    return out

Documentation: Function Documentation Syntax

#' Round a number *up* to a
#' certain number of digits.
#'
#' @param x (double) The value
#'   to be rounded.
#' @param dig (int) The
#'   number of digits to round
#'   to.
#'
#' @return The rounded number.
#'
#' @examples
#' # This returns `2.15`
#' round_up(2.141, 2)
round_up <- function(x, dig) {
    f <- 10 ** dig
    out <- ceiling(x * f) / f
    return(out)
}

def round_up(x, dig):
    """
    Round a number *up* to a
    certain number of digits.

    Parameters
    ----------
    x : float
        The value to be
        rounded.
    dig : int
        The number of digits to
        round to.

    Returns
    -------
    float
        The rounded number.

    Examples
    --------
    >>> round_up(2.141, 2)
    2.15
    """
    f = 10 ** dig
    out = np.ceil(x * f) / f
    return out

Documentation: Function Documentation Syntax

#' Round a number *up* to a
#' certain number of digits.
#'
#' @param x (double) The value
#'   to be rounded.
#' @param dig (int) The
#'   number of digits to round
#'   to.
#'
#' @return The rounded number.
#'
#' @examples
#' # This returns `2.15`
#' round_up(2.141, 2)
round_up <- function(x, dig) {
    f <- 10 ** dig
    out <- ceiling(x * f) / f
    return(out)
}

def round_up(x, dig):
    """
    Round a number *up* to a
    certain number of digits.

    Parameters
    ----------
    x : float
        The value to be
        rounded.
    dig : int
        The number of digits to
        round to.

    Returns
    -------
    float
        The rounded number.

    Examples
    --------
    >>> round_up(2.141, 2)
    2.15
    """
    f = 10 ** dig
    out = np.ceil(x * f) / f
    return out

Documentation: Function Documentation Syntax

#' Round a number *up* to a
#' certain number of digits.
#'
#' @param x (double) The value
#'   to be rounded.
#' @param dig (int) The
#'   number of digits to round
#'   to.
#'
#' @return The rounded number.
#'
#' @examples
#' # This returns `2.15`
#' round_up(2.141, 2)
round_up <- function(x, dig) {
    f <- 10 ** dig
    out <- ceiling(x * f) / f
    return(out)
}

def round_up(x, dig):
    """
    Round a number *up* to a
    certain number of digits.

    Parameters
    ----------
    x : float
        The value to be
        rounded.
    dig : int
        The number of digits to
        round to.

    Returns
    -------
    float
        The rounded number.

    Examples
    --------
    >>> round_up(2.141, 2)
    2.15
    """
    f = 10 ** dig
    out = np.ceil(x * f) / f
    return out

Documentation: Function Documentation Syntax

#' Round a number *up* to a
#' certain number of digits.
#'
#' @param x (double) The value
#'   to be rounded.
#' @param dig (int) The
#'   number of digits to round
#'   to.
#'
#' @return The rounded number.
#'
#' @examples
#' # This returns `2.15`
#' round_up(2.141, 2)
round_up <- function(x, dig) {
    f <- 10 ** dig
    out <- ceiling(x * f) / f
    return(out)
}

def round_up(x, dig):
    """
    Round a number *up* to a
    certain number of digits.

    Parameters
    ----------
    x : float
        The value to be
        rounded.
    dig : int
        The number of digits to
        round to.

    Returns
    -------
    float
        The rounded number.

    Examples
    --------
    >>> round_up(2.141, 2)
    2.15
    """
    f = 10 ** dig
    out = np.ceil(x * f) / f
    return out

Documentation: Issues & PRs

Jumping now to our approach to what I’ll call GitHub < air quotes > “documentation” via issues and pull requests
If you’re on a data science team, there are probably a few different approaches to how you could structure your collaborative workflows
I’m going to show our current approach, which I believe is not only ours, but a similar workflow to what’s in place at other organizations we’ve seen, including Posit
First, a user or developer identifies a bug in the software or comes up with an enhancement that they are suggesting implement, and they write this up in a detailed “issue”
Once that issue has been created, when we’re ready to actually work on making the required code changes, we first create a new git branch in the repository specifically for the purpose of addressing that “issue”
Then they go off and write or modify the code that fixes the bug or adds the enhancement
And once the developer feels like the updated code is in a good spot, they can write up a Pull Request (or “PR”, for short) that details how they addressed the issue, what changed, and more

Documentation: Issues

Anatomy of a Good Issue

Overview

An overview of the issue or proposed enhancement, including rationale.

Reproducible Example

Some code that someone else can run to reproduce the issue or show the current shortfall that the proposed enhancement will overcome.

Potential Solution(s)

Discussion (possibly including code) regarding possible solution(s).

Documentation: Issues

Documentation: Pull Requests

Anatomy of a Good Pull Request

Overview

The purpose of the pull request and the associated Issue(s) it addresses.

Details

The technical aspects of how the Issue was addressed in code, as well as any design decisions that were made along the way and/or hurdles that were overcome.

How to Test

Provide instructions and code that the reviewer can run to see the impact of your changes.

Documentation: Pull Requests

Final Thoughts

Get In Touch With Me

ketchbrook.com

in/michaeljthomas2

@mike-thomas.bsky.social

mthomas-ketchbrook | ketchbrookanalytics