LiteSwitch Offline Test Set-up¶
Offline testing is now available to prospective LiteSwitch customers. During the test, the LiteSwitch recommendation model will analyze a static snapshot of your trading logs and produce baseline results for compression rate and associated media impact. You can use these results as a benchmark for performance in the real-time environment.
Getting started with offline testing is simple – only a 24 hour snapshot of trading data is required to power the test, and results are available in two weeks.
Getting Started¶
To perform the offline test, we’ll require log-level data for bid requests and impressions. Log files can be shared in any SaaS storage bucket solution you prefer and can give us access, or otherwise the LiteSwitch team will assist you in creating a repository to deposit logs. If creating repository is the preferable option, please provide:
Preferred Cloud Provider: AWS or GCS
Name of the service account or resource name that will be utilized for access.
The LiteSwitch team will reach out once the bucket is ready. Log requirements are as follows:
Fields:
Bid Request: raw OpenRTB request body, including wseat array (fields described in this document)
Impression: imp_id, bid_id, dsp_id and price (fields described in this document).
Timeframe: 24 hour snapshot of trading. Request and Impression timeframes should align for bid context matching purposes.
Filetype: .avro preferred; .jsonl, .txt and .csv also accepted
Format:
Bid Request and Impression logs should be submitted in two separate files.
All strings (either bid requests or impressions) should be separated into different lines of the file.
Impression logs may be submitted in string format (see the sample data example below for reference) OR with parameters split into columns.
Sampling:
Bid Request logs CAN be sampled. If this data is sampled, please share the ratio with the LiteSwitch team. The ratio should be a fixed number < 100; please inform the LiteSwitch team whenever the sampling ratio changes.
Impression logs CANNOT be sampled. Impression logs must include at least 100,000 impressions attributed to bid requests. Impressions not attributed to bid request cannot be used for testing.
Please get in touch with the LiteSwitch team if you are not able to comply with any of these requirements.
Log Examples¶
Example of bid request log (raw bid requests with dsp_id field)
{
"id": "1688216407494039-70",
"imp": [
{
"id": "1",
"video": {
"w": 480,
"h": 320
},
"tagid": "33709315",
"pmp": {
"deals": [],
"private_auction": 0
},
"secure": 1
}
],
"app": {
"id": "28168874",
"bundle": "com.example",
"publisher": {
"id": "1157777",
"name": "myCast TV LTD"
}
},
"device": {
"ua": "Mozilla/5.0 (Linux; Android 10; SM-S111DL Build/QP1A.190711.020; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0Chrome/112.0.5615.135 Mobile Safari/537.36",
"ip": "68.44.228.91",
"geo": {
"country": "USA"
},
"ipv6": "::ffff:68.44.228.91",
"language": "en",
"model": "SM-S111DL",
"devicetype": 4,
"ifa": "82c27baa-b713-439e-9cd2-07e686ab3632"
},
"user": {
"id": "82c27baa-b713-439e-9cd2-07e686ab3632"
},
"dsp_id": [
"25746"
],
"ext": {
"seat_user_matched": [
"25746"
]
}
}
{
"id": "1688216411184000-253",
"imp": [
{
"id": "1",
"video": {
"w": 1280,
"h": 720
},
"tagid": "8264241",
"pmp": {
"deals": [],
"private_auction": 0
},
"secure": 1
}
],
"app": {
"id": "123456",
"bundle": "com.example",
"publisher": {
"id": "971985",
"name": "XUMO, LLC"
}
},
"device": {
"ua": "Mozilla/5.0 (Linux; x86_64 GNU/Linux) AppleWebKit/601.1 (KHTML, like Gecko) Version/8.0 Safari/601.1 WPE AAMP/4.6",
"ip": "68.44.228.91",
"geo": {
"country": "USA"
},
"ipv6": "2001:558:501d:23:6c66:7f04:9935:175d",
"language": "en",
"devicetype": 3,
"ifa": "893c992d-0588-49c7-b1ac-7ce0b02c3c4b"
},
"user": {
"id": "893c992d-0588-49c7-b1ac-7ce0b02c3c4b"
},
"dsp_id": [
"717", "432"
],
"ext": {
"seat_user_matched": [
"717", "432"
]
}
}
Example of impression logs (as a string or with parameters separation):
/***_imp?bid_id=1688216600595007-372&price=0.95761786400748&dsp_id=30833&imp_id=1
/***_imp?bid_id=1688216606472078-251&price=2.077077071199552&dsp_id=717&imp_id=1
Example of impression logs, .jsonl
{"bid_id": "80b4e3e1-eaad-4c96-aa6c-1f5dabe85832", "dsp_id": "1234", "imp_id": "1233de6b4483b84a", "price": 5.678}
{"bid_id": "0b8e8cda-d372-4b57-8df0-94e6acfa2afb", "dsp_id": "5678", "imp_id": "5", "price": 1.234}
Example of impression logs, .csv
bid_id,dsp_id,imp_id,price
80b4e3e1-eaad-4c96-aa6c-1f5dabe85832,1234,1233de6b4483b84a,5.678
0b8e8cda-d372-4b57-8df0-94e6acfa2afb,5678,5,1.234
Log Files Validation¶
Before uploading bid and imp logs to the bucket, please make sure that the files format is valid. It’s especially important for JSON Lines and CSV. To validate uploaded files you can use open-source utilities.
JSON Lines can be validated with jq utility.
Example for one file validation (replace
filename.jsonlwith your file name):jq empty filename.jsonlCSV files can be validated with awk utility.
Example for imp files validation (replace
filename.csvwith your file name, comma is the default separator, 4 number of columns)
awk -F',' '
NF<4 { err="not enough fields" }
NF>4 { err="too many fields" }
/^,|,$/ { err=(err == "" ? "" : err " and ") "empty fields" }
err != "" { print err, "at line", NR | "cat>&2"; exit 1 }
' filename.csv
To validate all files in the directory, use the following script (use -h, --help to see all options):
./validate.sh /path/to/directory jsonl
See the script source code below.
#!/bin/bash
files_dir=$1
file_extension=$2
if [ "$files_dir" == "-h" ] || [ "$files_dir" == "--help" ]
then
echo "This script requires installed jq utility."
echo "Usage: `basename $0` [/path/to/logs/directory]"
exit 0
fi
if [ ! -d "$files_dir" ]; then
echo "Directory $files_dir does not exist."
exit 1
fi
if [ "$file_extension" != "jsonl" ] && [ "$file_extension" != "csv" ]; then
echo "Invalid file extension. Please enter jsonl or csv."
exit 1
fi
# Iterate over each file in the directory
for file in ${files_dir}/*.${file_extension}; do
filename=$(basename "$file")
if [ "$file_extension" == "jsonl" ]; then
ERROR="$(jq empty $file 2>&1)"
else
ERROR="$( (awk -F',' '
NF<4 { err="not enough fields" }
NF>4 { err="too many fields" }
/^,|,$/ { err=(err == "" ? "" : err " and ") "empty fields" }
err != "" { print err, "at line", NR | "cat>&2"; exit 1 }
' $file) 2>&1)"
fi
if [[ ! $ERROR ]]; then
echo "$filename is valid"
else
echo "$filename is NOT valid: $ERROR"
fi
done
What You Get Back¶
In two weeks, you will receive a report and a log file outlining the results of the offline test, including filtering performance and associated trading impact. The team will get in touch to review results, answer any questions and determine next steps for advancing the LiteSwitch partnership.
The report includes the following information:
DSP ID
Bid Requests
Impressions
Media Cost
Recommendations
Bid Request Filtration %
Media Cost Reduced %
You can find a report example in this file.
The log file includes the following information:
Bid_id
Imp_id
DSP ID (Wseat)
Recommendation (0/1)
Frequently Asked Questions¶
Q: How long does the offline test take?
A: The expected time frame for the entire end-to-end offline testing is about two weeks.
Q: How does the offline test work?
A: After the Supplier provides the required bid request and impression logs, the LiteSwitch team:
parses the bid request data into the LiteSwitch log format
matches impression logs to bid request logs
creates predict aggregates and builds an offline predict model
scores the provided logs using the offline model
creates a report with bid filtering metrics based on the bid request scores
Q: What performance can I expect to see from this offline test?
A: You can expect to see a minimum 50% compression rate for the offline test. Offline test results should be treated as a floor for performance in a live environment, where optimal performance would be achieved.
Q: How much does it cost to run an offline test?
A: The offline test is FREE and comes at no additional cost to you as a current BidSwitch client.
Q: Do I need to sign an addendum to run this test?
A: No additional paperwork is required for current BidSwitch clients, as the use of standard request and impression logs is covered in the existing contract.