Skip to content

Metabase: Steam API Documentation

You are here:
Estimated reading time: 3 min

Steam Data Automation

This script automates the extraction and processing of Steam data from various sources using Selenium for web scraping and MySQL for data storage. It downloads CSV files containing Steam game analytics and inserts the data into corresponding MySQL tables.

Installation

1. Ensure you have Python installed on your system.
2. Install the required Python packages using pip:
pip install -r requirements.txt
3. Download the appropriate ChromeDriver executable from the [official site](https://chromedriver.chromium.org/downloads) and ensure it’s in the same directory as the script.

Usage

1. Open a command prompt and navigate to the directory containing the Google Chrome app.
2. Run the Chrome browser with remote debugging enabled by executing:
chrome.exe --remote-debugging-port=9222
3. Log in to your Steamworks admin account in the opened Chrome browser and ensure that the active page is the partner portal URL: https://partner.steampowered.com/nav_games.php.
4. Open a command prompt and navigate to the directory containing the script.
5. Active virtual environment: `venv\Scripts\activate`
6. Run the script using:
python steam_downloader.py

Workflow

1. Initialization: The script loads Steam data configurations from a JSON file and initializes variables.
2. Downloading Data: It iterates over each Steam game and downloads CSV files containing sales, wishlists, traffic, and other analytics data.
3. Processing Data: The downloaded CSV files are processed, and empty files are handled gracefully. Data is transformed into lists and prepared for insertion into MySQL tables.
4. Extracting Data: Data is inserted into corresponding MySQL tables, and table creation is handled if tables don’t exist already.
5. Scheduling: The script runs on a daily schedule, fetching yesterday’s data and updating MySQL tables.

JSON File Configuration

Overview

The JSON file provided configures the data retrieval and storage process for Steam Partner reports. It includes details about games, sales, wishlists, wishlist cohorts, traffic, UTM country, and UTM daily data. Below is a breakdown of each section and its corresponding configuration.

Games

  • id: Steam application ID.
  • name: Name of the game.

Sales

  • filename: Prefix for downloaded sales report filenames.
  • tablename: Name of the MySQL table to store sales data.
  • skiprows: Number of rows to skip in the CSV file.
  • columns: Column names and their corresponding MySQL data types.

Wishlists

  • filename: Prefix for downloaded wishlist report filenames.
  • tablename: Name of the MySQL table to store wishlist data.
  • skiprows: Number of rows to skip in the CSV file.
  • columns: Column names and their corresponding MySQL data types.

Wishlist Cohorts

  • filename: Prefix for downloaded wishlist cohorts report filenames.
  • tablename: Name of the MySQL table to store wishlist cohorts data.
  • skiprows: Number of rows to skip in the CSV file.
  • columns: Column names and their corresponding MySQL data types.

Traffic

  • filename: Prefix for downloaded traffic report filenames.
  • tablename: Name of the MySQL table to store traffic data.
  • skiprows: Number of rows to skip in the CSV file.
  • columns: Column names and their corresponding MySQL data types.

UTM Country

  • filename: Prefix for downloaded UTM country report filenames.
  • tablename: Name of the MySQL table to store UTM country data.
  • skiprows: Number of rows to skip in the CSV file.
  • columns: Column names and their corresponding MySQL data types.

UTM Daily

  • filename: Prefix for downloaded UTM daily report filenames.
  • tablename: Name of the MySQL table to store UTM daily data.
  • skiprows: Number of rows to skip in the CSV file.
  • columns: Column names and their corresponding MySQL data types.

Usage

  1. Modify the JSON file according to your game IDs, desired filenames, table names, column names, and data types.
  2. Ensure the steam_data.json file is located in the same directory as the Python script.
  3. Run the Python script to automate the data retrieval and storage process.
  4. This configuration allows for flexible customization of data retrieval and storage based on specific game IDs and reporting requirements.

Script Components

  • Selenium: Used for web scraping and automating browser interactions.
  • MySQL Connector: Facilitates connections to MySQL databases and execution of SQL queries.
  • pywinauto, pygetwindow: For managing browser windows and ensuring focus during automation.
  • schedule: Enables scheduling tasks at specified intervals.
  • pandas, numpy: Data manipulation and handling missing values in CSV files.
  • datetime, os, csv: Standard libraries for handling date-time, file operations, and CSV file processing.

Important Notes

  • Ensure Chrome browser is running with remote debugging enabled on port 9222.
  • The script assumes you’re logged in to the Steamworks admin account in the Chrome browser.
  • URLs for data extraction should be accessible via Steamworks partner portal.
  • Schedule the script to run daily to fetch yesterday’s data.

 

Attachments

Was this article helpful?
Dislike 0
Views: 12
Back To Top