Cultivate Labs Support Article

Cultivate Labs Support Articles

Cultivate Forecasts - Feature Overview

Forecasting Question Management

  • Supports Opinion Pool, Continuous, Multi-Time Period, and discussion-only question types
  • Ability to tag questions and filter questions list according to tag
  • Crowdsourced question development, allowing users to submit, discuss, and iterate on potential forecasting questions
  • Admin capability to resolve questions, including retroactive resolution dates and resolution notes/messaging, and undo mistaken resolutions
  • Forecaster suggested resolutions with admin review & approval
  • Admin capability to issue official question clarifications
  • Forecaster ability to request an official clarification
  • Support for custom "challenges" to group/categorize questions


Forecasting Experience

  • Multiple forecast input interface options, including sliders, buttons, text fields, and draggable distribution graphs
  • Customizable forecast reminder emails, allowing user to specify the frequency of notifications
  • Guided forecasting tutorial to introduce users to forecasting interface
  • "Presentation" mode with live forecast updating
  • "Market research" mode for streamlined forecast collection
  • Stale forecast prompts to encourage users to update old forecasts
  • Ability to hide crowd forecast from forecasters
  • Analysis section in each question to surface insight about the crowd forecast composition and direction of recent forecast updates


Personal Forecast & Performance Tracking

  • "My Forecasts" dashboard showing your current forecasts and personal statistics
  • "My Performance" dashboard showing your score and overall accuracy rank over time
  • Ability to compare your performance to the site's best forecaster, median forecaster, and others
  • Personal profile with overall, per-challenge, and per-question score details
  • Detailed per-question daily score breakdowns


Forecast Aggregation

  • Multiple cutting edge, research-inspired aggregation methods for calculating crowd consensus probability
  • Ability to automatically "decay" old, stale forecasts out of the crowd consensus
  • Weighting of individual forecasters according to past accuracy


Forecast Rationales & Commenting

  • Elicit forecast rationales for users to explain their reasoning behind forecasts
  • Forecast "pre-mortems" to encourage forecasters to assess potential gaps in their thinking
  • Threaded discussions, including email notifications for participants of the discussion
  • Per-question and site-wide discussion forums
  • Ability for users to flag objectionable comments and admin workflow to review flagged comments
  • Strict flagging mode to hide any comment immediately upon flagging
  • Ability for admins to define a list of "hotwords" or phrases that will warn the user that they might be submitting sensitive information
  • Ability to automatically flag comments that contain "hotwords"
  • Daily digest email to admins containing list of comments containing potentially sensitive content
  • Ability to hide comment text from all emails to prevent data leaks
  • Extract & display source links cited by in comments & rationales
  • Anonymous commenting
  • Ability to sort & filter comments based on comment quality, length, upvotes, and others

Registration & Authentication

  • Support for SAML-based single sign on
  • Facebook, Twitter, and Google integration for authentication & registration
  • Custom terms of service with required user acknowledgement on sign up
  • Restrict registration to users from a specified list of domains
  • Ability to require a custom registration code at sign up
  • Restrict registration to only directly invited users
  • Customizable "welcome" modal to present to users on their first site visit
  • Forecaster onboarding wizard with "practice" forecasting questions


Integrations

  • Specify social meta tags to supply images & descriptions to services such as Twitter cards, Facebook Open Graph, and Slack message unfurling
  • Support for custom Google analytics codes
  • Optional Mailchimp mailing list registration at user sign up


Admin

  • Sticky admin announcements with associated email notifications to the site's participants
  • Release notes to notify forecasters of new site features
  • User management section for updating user info or suspending users
  • Admin-selected "featured" questions, challenges, and users


Data, Reporting, & Analysis

  • Full-featured API for forecast submission & data extraction
  • CSV reports
  • Question infographics
  • Site usage statistics
  • Granular pageview tracking
  • User cohort features for analysing performance of different groups of forecasters
  • Executive Dashboard for monitoring crowd, cohort, and team forecasts as well as site activity
  • Ability to define a reporting timezone for any timestamps included in CSV reports


Hosting & Security (see our security overview here)

  • Platform is SOC2 Type II certified
  • Automatic account locking after multiple failed password attempts
  • All traffic encrypted in transit and data encrypted at rest
  • Support for 2-factor authentication
  • Regular 3rd party penetration tests
  • Configurable session timeouts, including optional screen-clearing if page is left open
  • Configurable password complexity requirements
  • Auto-scaling to meet traffic spikes


User Privacy

  • User emails hidden from all non-admin participants
  • GDPR-minded user data export and account deletion requests
  • One-click email unsubscribe links
  • Optional cookie consent modal


Teams

  • Ability to group users into teams for collaborative forecasting
  • Team-based leaderboards
  • Private team discussion areas
  • Team "request" features, allowing users to form their own team and add teammates, subject to admin approval


Social

  • Site-wide activity feed
  • Ability to "follow" other forecasters
  • Filter activity feeds to show only users you are following
  • Custom leaderboards vs. your followers and vs. forecasters following you
  • Ability to mention other users in comments
  • Both automated and custom-assigned badges
  • User uploaded avatar images or choose from pre-programmed library of avatar options


Scoring & Leaderboards

  • Brier scoring of forecasters, teams, and the crowd consensus
  • Relative Brier scoring (aka. Net Brier Points)
  • Ordinal scoring for "ordered bin" style questions
  • Forecasting seasons for tracking performance on a periodic basis
  • Customizable scoring settings, including settings for daily scoring cutoff time & timezone
  • Leaderboards for overall accuracy, per-challenge, per-question, per-season, and comment upvotes received
  • Ability to define a minimum number of forecasted questions to be eligible for leaderboards


CMS & Blogging

  • Create custom on-platform pages using WYSIWYG editor or direct HTML/CSS input
  • Full-featured on-platform blogging
  • Upload & link to custom files, including image, videos, and PDF's


Surveys

  • Create custom surveys to collect additional information from forecasters
  • Support for text, radio, dropdown, checkbox, and matrix style responses
  • Review & export survey responses
  • Email notifications when new survey submission is received
  • Administer a follow-up survey after forecasters submit a forecast¬†


Training

  • Upload custom training decks or videos for forecasters to complete
  • Add quizzes at various points during or after a training course
  • Ability for forecasters to track their progress through training courses


Translations

  • Available in English, German, French, Spanish, and Chinese (with the ability to add further languages as needed)
  • Questions, challenges, tags, and seasons can be input in multiple languages, allowing forecasters to select their preferred language


Interface Customization

  • Specify custom CSS to apply a theme to every page of the application
  • Admin-created custom "ad" tiles with full HTML support to tailored messaging
  • Add custom navigation links
  • Customizable question "intro" field
  • Upload custom site logo


Email notifications

  • Daily or weekly activity digest emails
  • Email notifications for various platform features, including question life-cycle (published, clarification, ending soon, resolution) notifications, badge awards, comment replies, and user mentions.
  • Ability to enable or disable any & all email notifications