LiteSwitch Offline Test Set-up

Offline testing is now available to prospective LiteSwitch customers. During the test, the LiteSwitch recommendation model will analyze a static snapshot of your trading logs and produce baseline results for compression rate and associated media impact. You can use these results as a benchmark for performance in the real-time environment.

Getting started with offline testing is simple – only a 24 hour snapshot of trading data is required to power the test, and results are available in two weeks.

Getting Started

To perform the offline test, we’ll require log-level data for bid requests and impressions. Log files can be shared in any SaaS storage bucket solution you prefer and can give us access, or otherwise the LiteSwitch team will assist you in creating a repository to deposit logs. If creating repository is the preferable option, please provide:

  • Preferred Cloud Provider: AWS or GCS

  • Name of the service account or resource name that will be utilized for access.

The LiteSwitch team will reach out once the bucket is ready. Log requirements are as follows:

  • Fields:

    • Bid Request: raw OpenRTB request body, including wseat array (fields described in this document)

    • Impression: imp_id, bid_id, dsp_id and price (fields described in this document).

  • Timeframe: 24 hour snapshot of trading. Request and Impression timeframes should align for bid context matching purposes.

  • Filetype: .avro preferred; .jsonl, .txt and .csv also accepted

  • Format:

    • Bid Request and Impression logs should be submitted in two separate files.

    • All strings (either bid requests or impressions) should be separated into different lines of the file.

    • Impression logs may be submitted in string format (see the sample data example below for reference) OR with parameters split into columns.

  • Sampling:

    • Bid Request logs CAN be sampled. If this data is sampled, please share the ratio with the LiteSwitch team. The ratio should be a fixed number < 100; please inform the LiteSwitch team whenever the sampling ratio changes.

    • Impression logs CANNOT be sampled. Impression logs must include at least 100,000 impressions attributed to bid requests. Impressions not attributed to bid request cannot be used for testing.

Please get in touch with the LiteSwitch team if you are not able to comply with any of these requirements.

Log Examples

Example of bid request log (raw bid requests with dsp_id field)

 {
  "id": "1688216407494039-70",
  "imp": [
      {
          "id": "1",
          "video": {
              "w": 480,
              "h": 320
          },
          "tagid": "33709315",
          "pmp": {
              "deals": [],
              "private_auction": 0
          },
          "secure": 1
      }
  ],
  "app": {
      "id": "28168874",
      "bundle": "com.example",
      "publisher": {
          "id": "1157777",
          "name": "myCast TV LTD"
      }
  },
  "device": {
      "ua": "Mozilla/5.0 (Linux; Android 10; SM-S111DL Build/QP1A.190711.020; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0Chrome/112.0.5615.135 Mobile Safari/537.36",
      "ip": "68.44.228.91",
      "geo": {
          "country": "USA"
      },
      "ipv6": "::ffff:68.44.228.91",
      "language": "en",
      "model": "SM-S111DL",
      "devicetype": 4,
      "ifa": "82c27baa-b713-439e-9cd2-07e686ab3632"
  },
  "user": {
      "id": "82c27baa-b713-439e-9cd2-07e686ab3632"
  },
  "dsp_id": [
      "25746"
  ],
  "ext": {
      "seat_user_matched": [
          "25746"
      ]
   }
}

{
  "id": "1688216411184000-253",
  "imp": [
      {
          "id": "1",
          "video": {
              "w": 1280,
              "h": 720
          },
          "tagid": "8264241",
          "pmp": {
              "deals": [],
              "private_auction": 0
          },
          "secure": 1
      }
  ],
  "app": {
      "id": "123456",
      "bundle": "com.example",
      "publisher": {
          "id": "971985",
          "name": "XUMO, LLC"
      }
  },
  "device": {
      "ua": "Mozilla/5.0 (Linux; x86_64 GNU/Linux) AppleWebKit/601.1 (KHTML, like Gecko) Version/8.0 Safari/601.1 WPE AAMP/4.6",
      "ip": "68.44.228.91",
      "geo": {
          "country": "USA"
      },
      "ipv6": "2001:558:501d:23:6c66:7f04:9935:175d",
      "language": "en",
      "devicetype": 3,
      "ifa": "893c992d-0588-49c7-b1ac-7ce0b02c3c4b"
  },
  "user": {
      "id": "893c992d-0588-49c7-b1ac-7ce0b02c3c4b"
  },
  "dsp_id": [
      "717", "432"
  ],
  "ext": {
      "seat_user_matched": [
          "717", "432"
      ]
   }
}

Example of impression logs (as a string or with parameters separation):

/***_imp?bid_id=1688216600595007-372&price=0.95761786400748&dsp_id=30833&imp_id=1
/***_imp?bid_id=1688216606472078-251&price=2.077077071199552&dsp_id=717&imp_id=1

Example of impression logs, .jsonl

{"bid_id": "80b4e3e1-eaad-4c96-aa6c-1f5dabe85832", "dsp_id": "1234", "imp_id": "1233de6b4483b84a", "price": 5.678}
{"bid_id": "0b8e8cda-d372-4b57-8df0-94e6acfa2afb", "dsp_id": "5678", "imp_id": "5", "price": 1.234}

Example of impression logs, .csv

bid_id,dsp_id,imp_id,price
80b4e3e1-eaad-4c96-aa6c-1f5dabe85832,1234,1233de6b4483b84a,5.678
0b8e8cda-d372-4b57-8df0-94e6acfa2afb,5678,5,1.234

Log Files Validation

Before uploading bid and imp logs to the bucket, please make sure that the files format is valid. It’s especially important for JSON Lines and CSV. To validate uploaded files you can use open-source utilities.

  • JSON Lines can be validated with jq utility.

    Example for one file validation (replace filename.jsonl with your file name): jq empty filename.jsonl

  • CSV files can be validated with awk utility.

    Example for imp files validation (replace filename.csv with your file name, comma is the default separator, 4 number of columns)

 awk -F',' '
   NF<4 { err="not enough fields" }
   NF>4 { err="too many fields" }
   /^,|,$/ { err=(err == "" ? "" : err " and ") "empty fields" }
   err != "" { print err, "at line", NR | "cat>&2"; exit 1 }
' filename.csv

To validate all files in the directory, use the following script (use -h, --help to see all options):

./validate.sh /path/to/directory jsonl

See the script source code below.

 #!/bin/bash
 files_dir=$1
 file_extension=$2
 if [ "$files_dir" == "-h" ] || [ "$files_dir" == "--help" ]
 then
   echo "This script requires installed jq utility."
   echo "Usage: `basename $0` [/path/to/logs/directory]"
   exit 0
 fi
 if [ ! -d "$files_dir" ]; then
   echo "Directory $files_dir does not exist."
   exit 1
 fi
 if [ "$file_extension" != "jsonl" ] && [ "$file_extension" != "csv" ]; then
   echo "Invalid file extension. Please enter jsonl or csv."
   exit 1
 fi
 # Iterate over each file in the directory
 for file in ${files_dir}/*.${file_extension}; do
   filename=$(basename "$file")
   if [ "$file_extension" == "jsonl" ]; then
     ERROR="$(jq empty $file 2>&1)"
   else
     ERROR="$( (awk -F',' '
       NF<4 { err="not enough fields" }
       NF>4 { err="too many fields" }
       /^,|,$/ { err=(err == "" ? "" : err " and ") "empty fields" }
       err != "" { print err, "at line", NR | "cat>&2"; exit 1 }
       ' $file) 2>&1)"
   fi
   if [[ ! $ERROR ]]; then
     echo "$filename is valid"
   else
     echo "$filename is NOT valid: $ERROR"
   fi
done

What You Get Back

In two weeks, you will receive a report and a log file outlining the results of the offline test, including filtering performance and associated trading impact. The team will get in touch to review results, answer any questions and determine next steps for advancing the LiteSwitch partnership.

The report includes the following information:

  • DSP ID

  • Bid Requests

  • Impressions

  • Media Cost

  • Recommendations

  • Bid Request Filtration %

  • Media Cost Reduced %

You can find a report example in this file.

The log file includes the following information:

  • Bid_id

  • Imp_id

  • DSP ID (Wseat)

  • Recommendation (0/1)

Frequently Asked Questions

  • Q: How long does the offline test take?

  • A: The expected time frame for the entire end-to-end offline testing is about two weeks.

  • Q: How does the offline test work?

  • A: After the Supplier provides the required bid request and impression logs, the LiteSwitch team:

    • parses the bid request data into the LiteSwitch log format

    • matches impression logs to bid request logs

    • creates predict aggregates and builds an offline predict model

    • scores the provided logs using the offline model

    • creates a report with bid filtering metrics based on the bid request scores

  • Q: What performance can I expect to see from this offline test?

  • A: You can expect to see a minimum 50% compression rate for the offline test. Offline test results should be treated as a floor for performance in a live environment, where optimal performance would be achieved.

  • Q: How much does it cost to run an offline test?

  • A: The offline test is FREE and comes at no additional cost to you as a current BidSwitch client.

  • Q: Do I need to sign an addendum to run this test?

  • A: No additional paperwork is required for current BidSwitch clients, as the use of standard request and impression logs is covered in the existing contract.