Custom JavaScript

Custom JavaScript lets you attach a script to a question in a human survey. The script runs in the respondent’s browser once the question is on the page, so you can build interactive tasks, measure timing and behaviour, and record events with the survey.

Custom JavaScript is available on approved accounts only. Write to [email protected] to request access. Until your account is approved, a humanize schema that includes a script is rejected when you save it, and no part of that schema is saved.

Checking access

from edsl import Coop

coop = Coop()
coop.get_custom_js_access()  # True once your account is approved

Access is per account: once yours is approved, the human surveys you create can include custom JavaScript.

Adding a script

Scripts go in the humanize schema, under questions[question_name].javascript.hooks. A hook maps an event to the script that runs on it. The only event is "question.ready", which fires once the question has been rendered and its inputs are on the page.

from edsl import Survey, QuestionMultipleChoice

survey = Survey([
    QuestionMultipleChoice(
        question_name="choice",
        question_text="Which option do you prefer?",
        question_options=["A", "B", "C"],
    ),
])

time_to_first_answer = """
const container = this.getQuestionContainer();
const shownAt = performance.now();

container.addEventListener("change", () => {
    ep.log("first_answer", { ms: Math.round(performance.now() - shownAt) });
}, { once: true });
"""

humanize_schema = {
    "questions": {
        "choice": {"javascript": {"hooks": {"question.ready": time_to_first_answer}}},
    },
}

survey.humanize(humanize_schema=humanize_schema)

To add or change a script on a survey that already exists, use Coop.patch_human_survey_humanize_schema:

coop.patch_human_survey_humanize_schema(
    "your-human-survey-uuid",
    {"questions": {"choice": {"javascript": {"hooks": {"question.ready": time_to_first_answer}}}}},
)

Set "javascript": None on a question to remove its script. Most question types support javascript, including survey messages; see Humanize schema for the list. compute and image_generation questions do not: they run in the background and never appear on the page.

Writing a hook

A hook is the body of a function, not a function. Write the statements you want to run; don’t wrap them in function () { ... }. Inside a hook you have:

Name	What it is
`this.getQuestionContainer()`	The question’s element on the page. Find your question’s inputs and add your own elements inside it.
`context.questionName`	The name of the question the hook is running for.
`context.questionType`	The question’s type, for example `"multiple_choice"`.
`context.isPreview`	`true` when the survey was opened through a preview link, which does not save a response.
`ep.log(eventName, payload)`	Records an event with the survey. See below.

context carries nothing that identifies the respondent. Each question’s hook runs once when that question appears. If the same script is attached to several questions, it runs once for each, and context.questionName tells them apart.

Recording events with `ep.log`

ep.log(eventName, payload) records an event against the respondent’s response. Use it for anything you want to analyse later: timings, clicks, which condition a respondent saw.

eventName must be a non-empty string. Names longer than 200 characters are cut to 200.
payload is optional and must be convertible to JSON: objects, arrays, strings, numbers, booleans and null. A payload that isn’t (a page element, an object that refers to itself) is not recorded, and a warning appears in the browser console.
A payload larger than 4,000 characters, measured as JSON, is not recorded.
Each response records at most 10,000 events.

ep.log returns nothing. Events are sent in batches in the background, so don’t wait on it.

What a script may do

Scripts can build interface inside the survey, read and write answers, control when respondents can move on, measure behaviour, randomise conditions, read parameters from the page URL, load images, audio, video and other media to show, and use the camera, microphone or location for a task the respondent is told about. A script can also load a well-known, widely used library, such as d3, Chart.js or lodash, from a public CDN like cdnjs, jsDelivr or unpkg, at a pinned version (for example https://cdn.jsdelivr.net/npm/[email protected]/dist/d3.min.js, not d3@latest). Your own code must still be plain JavaScript written in the script itself. A script can’t be saved if it:

Loads code from anywhere else: an unfamiliar or personal site, a paste site, a little-known or custom package, source hosted somewhere other than a named library, or a URL put together while the script runs.
Builds code at runtime, for example by passing a constructed string to eval, new Function, setTimeout or setInterval, or by inserting markup that contains a script.
Makes any request to Expected Parrot, or sends respondent data anywhere else. ep.log is how data leaves a script.
Reads cookies, tokens, credentials, or storage it didn’t create.
Is obfuscated or minified so that what it does can’t be read from the source. This applies to your script, not to a well-known library it loads.
Asks for passwords or payment details, or imitates a login or other dialog.
Collects anything unrelated to taking the survey, or uses the camera, microphone or location covertly.
Keeps running after the respondent has finished or left the survey.
Misleads or traps the respondent, for example by opening popups, preventing them from leaving, or hiding the survey’s own notices.
Uses the respondent’s device for something other than the survey.

If a script can’t be saved, no part of that humanize schema is saved, and the survey keeps what it had. If you think a script was refused by mistake, write to [email protected].

Limits

Each hook can be at most 64,000 characters, and all hooks on one question at most 256,000.
One save can add at most 25 different new scripts. A script used unchanged on several questions counts once, and scripts the survey already has don’t count. Save larger surveys in smaller batches.

Retrieving events

Use Coop.get_all_human_survey_events to get every event a survey has recorded, oldest first:

from edsl import Coop

coop = Coop()

events = coop.get_all_human_survey_events("your-human-survey-uuid")
respondent_events = [e for e in events if not e["is_preview"]]

Events from preview links are included and marked is_preview, so leave them out of an analysis. Events that arrive while the download runs are included, and no event comes back twice. To fetch new events later without downloading everything again, pass the id of the last event you downloaded as after. Take it from the full download, before filtering out previews:

last_id = events[-1]["id"] if events else None
new_events = coop.get_all_human_survey_events(
    "your-human-survey-uuid", after=last_id
)

Fetching only new events is useful for checking your scripts while a survey is collecting, but it can occasionally miss an event that was still being saved when you downloaded. For analysis, download the full log once fielding has ended, at least an hour after the last response.

To fetch one batch at a time, use Coop.get_human_survey_events. It takes after and limit (at most 200, the default), and returns {"events", "next_cursor", "has_more"}. Pass next_cursor as after to get the next batch while has_more is True. next_cursor is returned even when there is nothing more, so you can keep it and use it later. To see how many events a survey has without downloading them, use Coop.count_human_survey_events. The count is as of the call; on a survey still collecting, a download started afterwards can return more. Each event includes:

Field	Description
`id`	The event’s unique id
`received_at`	When Expected Parrot received it (ISO 8601, UTC)
`client_ts`	When the respondent’s browser logged it (ISO 8601, UTC)
`response_uuid`	The response the event belongs to
`is_preview`	Whether the event came from a preview link
`log_session_uuid`	The page load the event was logged in. Each time a respondent opens or reloads the survey starts a new session, so one response can have several.
`session_state`	Whether that session’s log is whole. See below.
`client_seq`	The event’s position in its session, counting from 0. Sort by it to get the order events were logged in. A gap means an event in between was not recorded.
`question_name`	The question whose script logged it
`event_type`	What kind of event it is. Always `"ep.log"` for now.
`event_name`	The `eventName` passed to `ep.log`
`hook_event`	The hook that was running when the event was logged, for example `"question.ready"`
`source`	What logged the event. Always `"author_script"` for now.
`payload`	The `payload` passed to `ep.log`, or `None`

session_state is one of:

"complete": the session ended normally and no events are missing.
"truncated": some events from this session are known to be missing, for example because they broke the limits above.
"unknown": no events are known to be missing, but the session never signalled that it ended, for example because the respondent’s browser closed or lost its connection. Its log may or may not be whole.

session_state describes the whole session, so every event from one session carries the same value.

From the command line

The ep CLI covers the same steps. --javascript takes QUESTION:HOOK=FILE, naming the question, the hook, and a file holding the script:

# Check access
ep humanize custom-js-access

# Attach a script while building a schema, or to a survey that already exists
ep humanize schema create --survey survey.ep --javascript choice:question.ready=choice.js --output humanize.json
ep humanize schema set <human-survey-uuid> --javascript choice:question.ready=choice.js

# Remove a question's script
ep humanize schema set <human-survey-uuid> --clear-javascript choice

# Get the first batch of events, or save every event to a file
ep humanize events <human-survey-uuid>
ep humanize events <human-survey-uuid> --all --output events.jsonl

# Later, save only the events since the last download
ep humanize events <human-survey-uuid> --all --after <next_cursor> --output new.jsonl

# Count events without downloading them
ep humanize events <human-survey-uuid> --count

--all requires --output, which can end in .json or .jsonl. Every result includes next_cursor, the value to pass as --after next time. If a download fails part way, no output file is written. As with the Python methods, use --after for checking a survey while it collects, and a full download once fielding has ended for analysis.

Introduction

Getting Started

Core Concepts

Getting Data

Working with Results

Validating with Humans

No-code Apps

Platform

How-to Guides

Notebook Examples

Developers

Checking access

Adding a script

Writing a hook

Recording events with `ep.log`

What a script may do

Limits

Retrieving events

From the command line

​Checking access

​Adding a script

​Writing a hook

​Recording events with ep.log

​What a script may do

​Limits

​Retrieving events

​From the command line

Checking access

Adding a script

Writing a hook

Recording events with `ep.log`

What a script may do

Limits

Retrieving events

From the command line