Computing the State of JS user rank with Vercel Cron jobs

The State of JavaScript is an annual survey of the JavaScript ecosystem, to which I contribute as the maintainer of the Next.js surveyform app.

We ask questions about your favorite frameworks, features, tools or people to follow. We also run the State of CSS, State of HTML, State of React and State of GraphQL surveys with the same codebase.

The State of HTML 2024 is open, go take it!

We want to make the survey as fun to fill as possible. For instance, we include a user rank at the end of the survey. The more JavaScript features you know, the better your rank will be!

Details like this make your experience better. But a small feature doesn't always mean a small implementation.

Let's take a deep dive into how we implemented this user ranking system with Vercel Cron Jobs.

What's a rank?

The rank indicates how many developers knew less features than you. The smaller the better : everyone is in the top 100% while the top 0.1% is the holy grail. We suspect that only the people who actually write browsers and language specifications can reach the top 0.1%.

Anyway, from an infrastructure standpoint, to compute the rank we just need to count how many users knew fewer features than you, and how many users have answered the survey so far.

// Computing the rank this way on every response
// This is costly!
async function inefficientRank(knownFeatures: number) {
	const usersAbove = await db.users.count({ 
        knownFeatures: {$gte: knownFeatures} 
        })
    const totalUsers = await db.users.count({})
    return usersAbove/totalUsers
}

In theory, we need to update the rank every time a new user answers the survey. But in practice, this is way too costly, and not even very useful. When we reach 10,000 respondents, a single response has no significant impact on the ranks overall. We need at least a hundred new respondents to shift ranks from 1%.

Precomputing the quantiles

In order to make the rank computation cheaper, we precompute the ranks associated with each possible score. Then, when you answer the survey and claim to have used or heard of 21 features, we just need to retrieve the rank for this value of 21. We don't need to count the number of users below or above you again.

// We precompute the ranks regularly, 
// but not on every request
async function precomputeRank() {
    for (
        let knownFeatures = 0;
        knownFeatures < maxKnownFeatures;
        knownFeatures++
    ) {
		const usersAbove = await db.users.count(
            { knownFeatures: {$gte: knownFeatures} }
        )
    	const totalUsers = await db.users.count({})
        ranks[knownFeatures] = usersAbove/totalUsers
    }
	await db.ranks.store(ranks)
}
// Cheap rank computation on every request
async function rank(knownFeatures:number) {
    const ranks = await db.ranks.findLatest({})
    return ranks[knownFeature]
}

The real implementation is available in our open-source repository. It uses a MongoDB aggregation to compute the ranks.

Updating precomputed ranks on demand

Thanks to precomputation, we have solved our performance issue. But we have a new problem: how to update the precomputed ranks on a regular basis?

The typical solution is to craft an API endpoint protected by a key that is only known to the survey maintainers. Then we can call this API endpoint from our admin interface to update the ranks.

// "/api/precompute-ranks"
export const GET = async (req: NextRequest) => {
    // only survey admins know the secret key
    checkSecretKey(req);
    await precomputeRanks()
    return NextResponse.json(
        {message: "ranks have been updated"}
        )
}

The secret key validation is not an elaborate system; it's just there to prevent attackers from inflating our bills by triggering useless computations.

// we compare the key URL param 
// to a SECRET_KEY environment variable
export function checkSecretKey(req: NextRequest) {
    const key = req.nextUrl.searchParams.get("key");
    if (key !== process.env.SECRET_KEY) {
        throw new Error("invalid key")
    }
}

Great, we have a working, efficient system! We can find the precomputed ranks in our database.

But I am not satisfied yet. At Devographics, we are very lazy people! Having to remember to run an API call here and there is way more work than we can accept. Let's set up a cron job to automate the update task.

Updating ranks regularly with a cron job

A cron job is a computation that runs regularly on a timed interval. Thanks to cron jobs, we can be sure that our user ranks are kept up to date even during our sleep or when we play Dwarf Fortress.

Cron jobs are usually set at the OS level, so they can run reliably even when your app is not active. This is especially necessary in serverless environments where your application may not be running all the time.

We host the survey app on Vercel, which offers a cron job service. Vercel crons are specifically tailored to trigger API calls, so we can reuse the precomputation API endpoint created earlier.

// in vercel.json
// will trigger a request every day at 4am
"crons": [
    {
      "path": "/api/precompute-ranks",
      "schedule": "0 4 * * *"
    },

We need to adapt our security check to accommodate Vercel's approach. The cron job secret key has to be stored in an environment variable named "CRON_SECRET". When running the cron job, Vercel will inject this value into the Authorization header.

export function checkCronSecret(req: NextRequest) {
	const authHeader = req.headers.get("Authorization")
	return authHeader === `Bearer ${process.env.CRON_SECRET}`
}

And we are good to go!

To sum things up:

• Ranks for each possible score will be updated every day at 4am, instead of being computed for every user request
• To do so, a cron job fires a request to an API endpoint
• This endpoint is secured via an environment variable, so only admin users and cron jobs can trigger it

Want to learn more about Next.js programming patterns? Discover my new course, NextPatterns