Cultivate Forecasts - Feature Overview
Forecasting Question Management
- Supports Opinion Pool, Continuous, Multi-Time Period, and discussion-only
question types
- Ability to tag questions and filter questions list according to tag
- Crowdsourced question development, allowing users to submit, discuss, and
iterate on potential forecasting questions
- Admin capability to resolve questions, including retroactive resolution
dates and resolution notes/messaging, and undo mistaken resolutions
- Forecaster suggested resolutions with admin review & approval
- Admin capability to issue official question clarifications
- Forecaster ability to request an official clarification
- Support for custom "challenges" to group/categorize
questions
Forecasting Experience
- Multiple forecast input interface options, including sliders, buttons,
text fields, and draggable distribution graphs
- Customizable forecast reminder emails, allowing user to specify the
frequency of notifications
- Guided forecasting tutorial to introduce users to forecasting
interface
- "Presentation" mode with live forecast updating
- "Market research" mode for streamlined forecast
collection
- Stale forecast prompts to encourage users to update old forecasts
- Ability to hide crowd forecast from forecasters
- Analysis section in each question to surface insight about the crowd
forecast composition and direction of recent forecast updates
Personal Forecast & Performance Tracking
- "My Forecasts" dashboard showing your current forecasts and
personal statistics
- "My Performance" dashboard showing your score and overall
accuracy rank over time
- Ability to compare your performance to the site's best forecaster,
median forecaster, and others
- Personal profile with overall, per-challenge, and per-question score
details
- Detailed per-question daily score breakdowns
Forecast Aggregation
- Multiple cutting edge, research-inspired aggregation methods for
calculating crowd consensus probability
- Ability to automatically "decay" old, stale forecasts out of
the crowd consensus
- Weighting of individual forecasters according to past accuracy
Forecast Rationales & Commenting
- Elicit forecast rationales for users to explain their reasoning behind
forecasts
- Forecast "pre-mortems" to encourage forecasters to assess
potential gaps in their thinking
- Threaded discussions, including email notifications for participants of
the discussion
- Per-question and site-wide discussion forums
- Ability for users to flag objectionable comments and admin workflow to
review flagged comments
- Strict flagging mode to hide any comment immediately upon flagging
- Ability for admins to define a list of "hotwords" or phrases
that will warn the user that they might be submitting sensitive
information
- Ability to automatically flag comments that contain
"hotwords"
- Daily digest email to admins containing list of comments containing
potentially sensitive content
- Ability to hide comment text from all emails to prevent data leaks
- Extract & display source links cited by in comments &
rationales
- Anonymous commenting
- Ability to sort & filter comments based on comment quality, length,
upvotes, and others
Registration & Authentication
- Support for SAML-based single sign on
- Facebook, Twitter, and Google integration for authentication &
registration
- Custom terms of service with required user acknowledgement on sign
up
- Restrict registration to users from a specified list of domains
- Ability to require a custom registration code at sign up
- Restrict registration to only directly invited users
- Customizable "welcome" modal to present to users on their
first site visit
- Forecaster onboarding wizard with "practice" forecasting
questions
Integrations
- Specify social meta tags to supply images & descriptions to
services such as Twitter cards, Facebook Open Graph, and Slack message
unfurling
- Support for custom Google analytics codes
- Optional Mailchimp mailing list registration at user sign up
Data, Reporting, & Analysis
-
Full-featured
API for forecast submission & data extraction
- CSV reports
- Question infographics
- Site usage statistics
- Granular pageview tracking
- User cohort features for analysing performance of different groups of
forecasters
- Executive Dashboard for monitoring crowd, cohort, and team forecasts as
well as site activity
- Ability to define a reporting timezone for any timestamps included in CSV
reports
Hosting & Security (see our security overview
here)
- Platform is SOC2 Type II certified
- Automatic account locking after multiple failed password attempts
- All traffic encrypted in transit and data encrypted at rest
- Support for 2-factor authentication
- Regular 3rd party penetration tests
- Configurable session timeouts, including optional screen-clearing if
page is left open
- Configurable password complexity requirements
- Auto-scaling to meet traffic spikes
- User emails hidden from all non-admin participants
- GDPR-minded user data export and account deletion requests
- One-click email unsubscribe links
- Optional cookie consent modal
Teams
- Ability to group users into teams for collaborative forecasting
- Team-based leaderboards
- Private team discussion areas
- Team "request" features, allowing users to form their own team
and add teammates, subject to admin approval
Social
- Site-wide activity feed
- Ability to "follow" other forecasters
- Filter activity feeds to show only users you are following
- Custom leaderboards vs. your followers and vs. forecasters following
you
- Ability to mention other users in comments
- Both automated and custom-assigned badges
- User uploaded avatar images or choose from pre-programmed library of
avatar options
Scoring & Leaderboards
-
Brier scoring of forecasters, teams, and the crowd consensus
-
Relative Brier scoring (aka. Net Brier Points)
-
Ordinal scoring for "ordered bin" style questions
- Forecasting seasons for tracking performance on a periodic basis
- Customizable scoring settings, including settings for daily scoring
cutoff time & timezone
- Leaderboards for overall accuracy, per-challenge, per-question,
per-season, and comment upvotes received
- Ability to define a minimum number of forecasted questions to be
eligible for leaderboards
CMS & Blogging
- Create custom on-platform pages using WYSIWYG editor or direct
HTML/CSS input
- Full-featured on-platform blogging
- Upload & link to custom files, including image, videos, and
PDF's
Surveys
- Create custom surveys to collect additional information from
forecasters
- Support for text, radio, dropdown, checkbox, and matrix style
responses
- Review & export survey responses
- Email notifications when new survey submission is received
- Administer a follow-up survey after forecasters submit a
forecast
Training
- Upload custom training decks or videos for forecasters to
complete
- Add quizzes at various points during or after a training
course
- Ability for forecasters to track their progress through training
courses
Translations
- Available in English, German, French, Spanish, and Chinese (with
the ability to add further languages as needed)
- Questions, challenges, tags, and seasons can be input in multiple
languages, allowing forecasters to select their preferred
language
Interface Customization
- Specify custom CSS to apply a theme to every page of the
application
- Admin-created custom "ad" tiles with full HTML
support to tailored messaging
- Add custom navigation links
- Customizable question "intro" field
- Upload custom site logo
Email notifications
- Daily or weekly activity digest emails
- Email notifications for various platform features, including
question life-cycle (published, clarification, ending soon,
resolution) notifications, badge awards, comment replies, and
user mentions.
- Ability to enable or disable any & all email
notifications