Accessing Arbitrary Paths via String Dot Notation in JavaScript

10 May 2020

In this blog post, I’d like to talk about a fun problem I had at work.

Without getting too nitty-gritty, here’s the gist of a problem.

How do you write a function that receives a path, such as the.path.to.a.key, and correctly uses the dot notation to traverse an object and return the value of the key?

const path1 = 'foo.bar' // Should return { baz: 42 }
const path2 = 'foo.bar.baz' // Should return 42
const path3 = 'davis' // Should return 'ford'
const path4 = 'foo.going.even.deeper.now' // Should return 0
const path5 = 'foo.bar.invalid_path' // Should return undefined

const obj = {
  davis: 'ford',
  foo: {
    bar: {
      baz: 42,
    },
    going: {
      even: {
        deeper: {
          now: 0,
        },
      },
    },
  },
}

My solution simply split and then reduced the keys and returns the value, if I found it.

It’s a lot of lines of code, and not super readable, but it does work, and it is nicely documented (in my opinion).

/**
 * Solution 1: The one I wrote
 */
const getKeyValue = (obj, key) => {
  // Best case scenario: The key is a top level key
  if (obj.hasOwnProperty(key)) return obj[key]

  // If there's no keys in the object, don't bother
  if (Object.keys(obj).length === 0) return undefined

  // Split the name into keys
  const keys = key.split('.')

  // Reduce the keys
  const { value } = keys.reduce(
    (accum, k, i) => {
      // Don't process any more if we've already set break to true
      if (accum.break) return accum

      // If the last key exists in the object, we've found our answer!
      if (i === keys.length - 1 && accum.obj.hasOwnProperty(k)) {
        accum.value = accum.obj[k]
        return accum
      }

      // Does the obj have this key? If so, keep drilling down
      if (accum.obj.hasOwnProperty(k)) {
        accum.obj = accum.obj[k]
        return accum
      }

      // If we can't find the key, we've been given an invalid path
      // So set `break` to true to stop processing
      accum.break = true
      return accum
    },
    // Initialize our `reduce` loop with this object
    {
      break: false,
      value: undefined,
      obj,
    }
  )

  return value
}

For reference, here is lodash’s implementation.

import castPath from './castPath.js'
import toKey from './toKey.js'

/**
 * The base implementation of `get` without support for default values.
 *
 * @private
 * @param {Object} object The object to query.
 * @param {Array|string} path The path of the property to get.
 * @returns {*} Returns the resolved value.
 */
function baseGet(object, path) {
  path = castPath(path, object)

  let index = 0
  const length = path.length

  while (object != null && index < length) {
    object = object[toKey(path[index++])]
  }
  return (index && index == length) ? object : undefined
}

When I asked some fellow developers at work for their ideas, I got some amazing answers! I’d particularly like to highlight this one.

Succint, to-the-point, and that break really helps performance-wise!

/**
 * Co-worker's suggestion
 */
const getKeyValue = (obj, path) => {
  const keys = path.split('.')
  while (keys.length) {
    let loc = keys.shift()
    if (obj.hasOwnProperty(loc)) {
      obj = obj[loc]
    } else {
      obj = undefined
      break
    }
  }
  return obj
}

You can check the performance of these two approaches in this Codepen.

Given 4 paths of varying complexity, and 1 incorrect key, here are the results:

• My approach takes 0.25ms
• My co-worker’s approach takes only 0.19ms!
  • That’s a 24% difference!

Long story short - always lean on people smarter than you when trying to develop fast code! :)

Creating a React/Redux JupyterLab Extension

24 Apr 2020

In this blog post, I’d like to demonstrate how to create a demo JupyterLab extension that leverages React and Redux (along with Typescript).

This is what we’ll end up with:

Documentation of React-enabled JupyterLab widgets is scarce and not always up to date. I hope to rectify that with this post. We’re going to do the following:

1. Set up our dev environment
2. Run cookiecutter to create the basic structure and dependencies of a new extension.
   • You don’t need to complete this step if you’re using my repository, this is already done for you.
3. Set up React-specific parts of the app

Of course, if you’d rather learn by reading code, just go ahead and skip to my Github repository.

Getting Started

By the way, many of these steps are directly pulled from this tutorial on creating extensions. If you’re unfamiliar with JupyterLab Extensions, this tutorial is a great resource!

If you don’t use conda, you can achieve similar results with other virtual environment managers. See here for other installation methods

We’re going to use conda to manage all of the our Python dependencies.

We’re going to create a new conda environment called jl-extension-env

conda create -n jl-extension-env -c conda-forge jupyterlab cookiecutter nodejs git

We can activate and deactivate this environment like so (we’ll automate this using iTerm2 in a moment.)

# Activate this environment when working on this repository
conda activate jl-extension-env
# Deactivate it otherwise
conda deactivate

Running cookiecutter

Note: skip this step if you’re just copying my repository as a starting point. This has already been done for you.

Run the following command wherever you’d like to create your new project.

cookiecutter https://github.com/jupyterlab/extension-cookiecutter-ts

Answer the prompts and then navigate into the newly created directory.

Installing the extension locally

Let’s install dependencies and link our development version of the extension with JupyterLab.

# Install dependencies and build the Typescript source
yarn && yarn build
# Link your development version of the extension with JupyterLab
jupyter labextension link .

You can watch the source directory for changes. Use the following commands.

yarn watch
jupyter lab --watch

Or… try my script to automatically open two tabs and run watch mode in both of them.

Working with React and Redux

Okay, so the gist of how a JupyterLab extension works is this.

There is an extension object that runs everything (located in index.ts). A MainAreaWidget is created, and we will attach our React app to that widget by extending JupyterLab’s ReactWidget.

We create a new class called ReactAppWidget, which extends ReactWidget. We override the render function and set up our <Provider> (which will handle our Redux store), and we render our <AppComponent>.

Note: I haven’t experimented with Context yet in the JupyterLab setting, but theoretically you would wrap your ContextProvider around <AppComponent /> here as usual.

That’s pretty much it! <AppComponent /> can be any valid React that you’d like - this render function is similar to a normal React app’s index.tsx entry point.

I’ve used redux-toolkit to manage the application’s store - if you’re not familiar with redux-toolkit, it is essentially the CreateReactApp of Redux. It provides a set of boilerplate best practices to get your project hooked up and running quickly. I personally love this library.

Go forth and develop!

You’re all ready to go! Start both watch modes, and you’re in business!

Once JupyterLab opens in your browser, click the “Commands” icon, and search for “Sample React Redux Extension”.

You’ll then see this neat app!

Congratulations, you’re ready to begin working on your very own React/Redux JupyterLab extension!

I hope this guide has been helpful. If you have any issues getting this to work, please open an issue on my Github repository.

Troubleshooting and other issues

• There is some funky behavior related to closing and re-opening the extension in the same session.
  • It is probably 100% my fault. I’ll fix the repo if/when I figure it out.
  • For now, this is left as an exercise for the reader :)
• Sometimes watch mode doesn’t seem to refresh very well. Try manually refreshing your page in the browser after making changes.
  • If that doesn’t work, just close your JupyterLab tabs and re-open them,
• If you get build errors when uninstalling the extension locally, try unlinking it as well
  • jupyter labextension unlink [your-extension-name]

AoS Reminders - 2019 in Review

31 Dec 2019

It’s the day before 2020 kicks in, and I’d like to take this opportunity to talk about the last 7 months of working on AoS Reminders.

I wrote the first iteration of AoS Reminders in May of 2019. I first developed it for my army (Seraphon), and then started to expand the tool to armies that my local gaming group played in order to get feedback.

Here’s the original thread on Lustria Online that kickstarted this whole adventure. Someone had posted looking for a Seraphon Cheat Sheet and I replied:

"I was also thinking of writing an online tool. I think it would be neat to input your list and have it spit out a checklist for every phase. Setup, post setup, start of hero phase first turn, etc

Remembering to roll constellations, when summoning occurs for Slann vs EOTG, when to place the blot toad, when to use certain command abilities... could be a fun weekend project!"

Fun weekend project - yeah right! Seven months, hundreds of thousands of lines of code, and 640 commits to the codebase later, I think it’s safe to say that this project got a little bigger than I initially expected.

Key Stats

• Launch date: Friday, May 17, 2019
• Over 100,000 pageviews since launching
• Over 40,000 unique users from all over the globe

• Over 300 subscribers!
  • Thank you to every single one of you for supporting me!
• Over 265,000 individual selections have been tracked - everything from factions, units, endless spells, and more.
  • 181,000 of these data points have occurred in the last month due to our expanded tracking - this number will rise quickly!
• 20 volunteer contributors to our codebase on Github

• 3,500+ PDF/HTML file imports
• 12,000+ Reminder PDFs generated/downloaded
• 45,000+ faction selections

Key Lessons

• You can’t do a project of this size solo
  • I don’t think this project would be successful without the valuable contributions of 20 hard-working volunteers
• Error logging and error tracking is more valuable than tracking successes
• I didn’t realize how beholden I would feel to subscribers once I introduced a payment model
  • Most of the pressure is in my own head, but I do feel intense pressure to make my subscribers happy.
• My girlfriend, Amanda, is a saint for putting up with seven months of me focusing on this thing!

What would I do differently?

Tracking individual selections

I didn’t track individual selections until late November, and I think that’s probably my #1 regret.

I was tracking faction selections, which was cool and useful, but I had no idea what was being selected other than that.

Once I worked with AoS Coach on getting some advanced stats together, I realized that I needed to overhaul my reporting system to spit out individual selections. We now track pretty much everything that occurs on the site, which is great, but it means that we don’t have any significant user data before December 2019.

I should have written the verify script earlier

I wrote a script in December that goes through every rule in the codebase and checks for certain wording to see if we’ve got it classified correctly.

For example, if a rule contains the phrase "In your hero phase, [...]", and the rule is currently shown DURING_GAME, the script will say "Rule X should probably be in DURING_HERO_PHASE"

Once I wrote this script, I had to update about 60 rules that were misclassified. In retrospect, I should have focused on creating this script early on to help contributors double-check their submissions.

I should have handled errors better in the early days

When I first started, I emphasized collecting data that would be considered “successful” - for example, I track when users click certain buttons or select armies, etc.

As the codebase and feature count grew, I realized I needed more information on what wasn’t working for users.

Examples of this include:

• When a user imports a PDF or HTML file to the tool, and there is an error parsing the file, I send a copy of the file to storage so that I can debug and fix the problem later.
• I fire off failure events to Google Analytics to make sure I’m aware of errors as they happen
• I provide numerous channels for users to get in touch with me in case of failure - Twitter, Discord, Github, a subreddit, and my email.

Things I did correctly

Tooling and tests

One thing I will credit myself with is our testing and tooling capabilities. Contributing to a new codebase can be very intimidating - I’ve tried my best to make it painless.

• We currently have over 300 automated tests in the codebase.
• We use linters and automated code stylers to enforce consistency in programming styles
• Our Github repo automatically runs tests, builds, etc
• We have an automated pipeline for deploying to production
• I’ve written detailed README.md and CONTRIBUTING.md documents for newcomers to the codebase

Encouraging feedback

As I’ve mentioned above, I’ve given users plenty of chances to provide me with feedback. I’d like to think I’m pretty responsive to users who have suggestions or problems.

Out of the 353 issues opened on Github, we’ve closed 330

I’ve spent countless hours answering emails, Reddit posts, and questions in Discords. I am pretty proud of how responsive we are!

Supporting the competitive scene

I’ve handed out free subscriptions to a bunch of Age of Sigmar tournaments all around the world. I think it’s win-win - I come from an e-sports background, and I know how motivating any sort of prize can be. I think that the AoS scene is still fairly small (but growing!)

Throwing in a hundred bucks worth of subscriptions to any given tournament is (in my opinion) the least I can do. It brings exposure to AoS Reminders (which makes me and my contributors happy), it rewards people who are actually playing the game at a high level, and it helps tournament organizers hype up their events!

Working with content creators

I’ve messaged, cajoled, and outright begged a heck of a lot of people in the past few months. I’ve pushed myself out of my normal comfort zone to talk to strangers and advocate for my product.

I strongly believe that AoS Reminders is a must-use application for Warhammer players, and that gives me the confidence to message random people and say “Hey, would you mind helping get the word out about this?”

In particular, I’d really like to thank the Rolling Bad crew, Re-Rolling Ones (Hi Jack!), Doom and Darkness, AoS Coach, 2+ Tough, Mr. Mephisto, Garagehammer, Realm & Ruins, BWG Sean, The Honest Wargamer, LLV, and a bunch of other people!

Two of my favorite videos to come from content creators:

AoS Coach’s excellent Emerging Meta series, where he uses AoS Reminders data to examine the meta of the Age of Sigmar scene. Get excited for next month’s edition, where he’ll have a lot more data to work with.

And 2+ Tough’s Gittin Good at AoS Real Fast video, where he walks through the site and talks about how it can help newer players enter the game.

Random Thoughts

List building became easier

I started 2019 as a Seraphon player, but I’m going into 2020 as an Ogor Mawtribes player.

I am going to Adepticon in 2020, so I’ve been theory-crafting Ogor lists.

Let me tell you a secret about list building… when you have 250k+ data points sourced from the AoS community, picking strong lists becomes quite a bit easier :)

Whenever I’m trying to figure out artifacts, traits, or spells to bring with my armies, I sneak a quick peek at the AoS Reminders stats for the last month, and I totally steal the most popular suggestions.

React and Typescript is the correct tech stack

I’m passionate about React (I attended React Conf 2018 and 2019 in Las Vegas), and I obsessively try to convert everyone I know to using Typescript.

I use both technologies at work, so it was natural to bring them to AoS Reminders.

I think that this project really demonstrates how powerful Typescript is, especially when working with remote programmers who live all over the world. Typescript’s ability to solidify data structures and create a self-documenting codebase is invaluable.

React continues to be an absolute stud for working with web apps, enough said.

The books are getting better

I read a lot of battletomes, a lot of warscrolls, and a lot of rules.

I can unequivocally say that the quality of rules-writing has improved so much in the second half of 2019. Books written in 2017-2018 are definitely a different breed - they often have strange rules, useless rules, or badly-worded rules.

By contrast, the 2019 books have been mostly superb, with great synergies and strong, well-written rules.

Obviously, some books are a little too strong (Hi, Ossiarch Bonereapers!), but I still would like to commend the GW team for really upping their battletome game in the latest releases!

What does GW think?

I’d love to talk to Games Workshop about getting early access to battletomes and anything else that would affect rules. Ideally, I’d like to be able to update the website as soon as a new battletome is released. As it stands now, we have a 1-2 day period where we lag behind the GW release schedule.

I’d also love to hear their thoughts on the site as a whole!

What’s next?

Good question :)

As always, we will continue adding new battletomes, FAQs, erratas, etc. to the tool.

We will be knocking out a few features - including Game Mode, which will be released soon - once you’re ready to play, you’ll be able to hide all of the unit selections stuff and just see the reminders that you need!

I am traveling to Adepticon 2020 in March :) If you see me there, come talk to me - I’ll give you some sweet AoS Reminders swag, and you can tell me what I could be doing better.

My goal at Adepticon is to win one game with my newly built Ogors!

Most of all, I want to continue making AoS Reminders the best tool it can possibly be. I want to continue making friends and building relationships with this amazing AoS community.

I get emails once in a while that tell me how much someone loves the tool, or telling me how they got their son or wife into the game using it. It warms my heart everytime someone sincerely thanks me for taking the time and effort to keep this tool going.

That feeling of warmth and pride, above all, will power my efforts in 2020.

Have a great New Year :)

Got a great idea for AoS Reminders that you think I should be working on? Message me on Twitter or post on the subreddit

AoS Reminders - October Stats Roundup

04 Nov 2019

Well, October was a huge month for AoS Reminders. We introduced dark mode, offline support, added Orruk Warclans, Cities of Sigmar, Ossiarch Bonereapers, and Ogor Mawtribes, and added support for Battlescribe army import. No sweat 😅

We were featured on Rerolling Ones, I was interviewed for over an hour on Realm and Ruin, and we even got a shoutout on Mr. Mephisto’s Rantcast.

Most of all, we continued to improve our library of rules. Our site is faster than ever, did I mention it works offline now, and our arsenal of tests continues to grow.

But have you ever wondered who else is using AoS Reminders?. I’m going to give you the army stats you’re here for, of course, but I’ll also talk a little bit about the audience for this website.

First things first - army stats

Stats for this month are going to be a little crazy! What I’ve observed in the past is Stormcast Eternals maintaining a strong lead over every other faction.

That is not the case anymore.

In October 2019, there were about 5,200 army selections made on the site.

Cities of Sigmar and Orruk Warclans absolutely dominated Stormcast Eternals.

I can already hear you saying - “Wait, what about Ossiarch Bonereapers and Ogor Mawtribes?”

They were added at the very end of October, so they won’t show up in the monthly stats, but if we condense our queries to the last few days (Oct 30 - Nov 4), we can see that they are pretty freaking popular.

Every time I post these pie charts, I get beat up for not defining what “Others” means - so here you go, enjoy! This is a full table of all of the selections made in October.

You can add allies to your army in AoS Reminders (although it seems like an underused feature - or maybe most people just aren’t using allies)

Import Usage

AoS Reminders is capable of taking in Azyr/Warscroll Builder PDFs or Battlescribe HTML files and instantly populating the rules you need for your list. Which is pretty freaking cool.

But who’s using what, and how much? *

* Keep in mind Battlescribe support was only added near the end of October, so it won’t have the same presence as Azyr/Warscroll Builder.

User Demographics

We had 4,500 unique users in October. Where are they all coming from, and what do they look like?

Our users are heavily concentrated in North America and the UK. Not a huge surprise there - GW is based in the UK, and I’m from NA and probably talk to a lot more Americans than anything else.

Australia has been popping up more and more lately, which I credit to Doom and Darkness and AoS Coach!

To my 37 users from New Zealand - cheers! I was there for a couple weeks in March and absolutely fell in love with your country.

Our users are overwhelmingly male. I think that is probably just representative of the hobby. I would like to see more women using the site!

One thing that surprised me is the age demographics. I really thought it would trend older (mostly 40s), but ~65% of our users are between 25-34. I guess that makes sense though - that’s the age where we start having disposable income, but not old enough to feel the existential dread of wasting our life playing with toy soldiers :)

Tech

When I first wrote the site, I believed that most of my users would use a desktop computer, because that’s how I use it. I was very surprised to see how many mobile users I have - and Android phones are majorly represented!

62% of our users are using mobile devices, and 40% of those are on Android. Since I’ve been tracking these stats, I have put a TON of work into improving mobile performance and UI, and I think I’ve been doing a decent job of that.

Did I mention the site works offline? How freaking cool is that? You can use it on the subway or in a plane!

Chrome absolutely towers over all other browsers in terms of usage, which isn’t really surprising - but poor Firefox, I really thought it had more market share than a paltry 4.6% :(

Caveat

These stats only track people who don’t use ad-blockers. Depending on which clickbait article you read, something like 30% of users run ad-blockers and therefore won’t be tracked.

I run an ad-blocker btw, so this isn’t a judgement :)

Closing Notes

It was a great month for AoS Reminders. We added a ton of features and content, and our users stay on the page for an average of 8 minutes, which is kinda crazy.

I’ve gotten so many kind words from people using the tool, and I’m excited to keep working on it and bringing you new features.

Ogor Mawtribes and Ossiarch Bonereapers are crazy popular right now, and Cities of Sigmar has clearly found a place in people’s hearts.

Something cool I’m working on right now - the ability to share armies with a URL.

So you could create an army list in AoS Reminders, click a “Share” button and get a link like aosreminders.com/?army=123abc, which you could send to a friend or post on a forum.

I think it’ll be pretty neat!

If you like this article, or the work I’ve done on this project, consider subscribing to support me. It would be a pretty cool thing to do!