GoReplay Setup for Testing Environments

Want to test your app with real production traffic without any risks? GoReplay is your answer.

Here’s what GoReplay does in simple terms:

• Captures live HTTP traffic from your production servers
• Replays that traffic to your test environments
• Lets you test new features with actual user behavior
• Doesn’t impact your production system at all

Quick Setup Guide:

1. Download GoReplay binary
2. Make it executable
3. Start capturing traffic with: sudo ./gor --input-raw :8000 --output-http http://staging.env

Key features you’ll get:

• Shadow testing: Test new code with real traffic
• Load testing: See if your app can handle production loads
• Traffic filtering: Focus on specific requests
• Security controls: Remove sensitive data automatically

Real Example: Videology used GoReplay to stream production traffic to QA environments for soak testing across different service versions.

Feature	Free Version	Pro Version
HTTP Traffic Capture	✓	✓
Multiple Endpoints	✓	✓
Binary Protocols	✗	✓
Cloud Storage	✗	✓
Enterprise Support	✗	✓

This guide shows you how to set up GoReplay, capture traffic, replay it safely, and integrate it into your testing workflow. You’ll learn practical tips for performance tuning and security best practices.

Related video from YouTube

Setup and Installation

Getting GoReplay up and running is a simple process that delivers strong traffic capture tools without unnecessary hassle. Its lightweight build makes it easy to integrate without major infrastructure adjustments.

Dependencies

Before you begin, ensure your system is ready with these basic needs:

• Root or sudo access to capture network traffic
• Enough storage for saving traffic data
• Network access to both source and target environments
• A supported Linux distribution or macOS (for network capture features)

If you’re working in a Kubernetes setup, make sure you have the following:

Component	Purpose	Requirement Level
Container Runtime	Running GoReplay pods	Required
Persistent Storage	Storing traffic data	Recommended
Network Policies	Allowing traffic mirroring	Required
Service Account	Pod permissions	Required

How to Install GoReplay

Installing GoReplay is a quick process. The easiest way is to download the latest binary from its official GitHub repository.

For Linux users, here’s how to get started:

1. Download the binary: Head to the official repository and grab the latest version.
2. Set permissions: Make the binary executable by adjusting its permissions.
3. Add to PATH: Move the binary to /usr/local/bin for system-wide use.

“GoReplay’s installation process is designed to be minimally invasive, requiring no changes to your production infrastructure beyond running the daemon on the same machine as your service,” according to the official documentation.

To ensure better security and performance, follow these tips:

• Use dedicated storage volumes for traffic data to avoid running out of disk space.
• Pre-configure network interfaces before starting the traffic capture.
• Set up proper permissions for the GoReplay binary to control access.
• Test the setup with a command like --input-raw :8000 --output-stdout to confirm everything works as expected.

GoReplay’s design makes it an easy fit for existing systems, catering to teams looking to enhance their testing workflows. It avoids unnecessary complexity, delivering a balance of straightforward setup and effective traffic capture and replay.

Using GoReplay to Record Traffic

GoReplay simplifies the process of capturing real-world user interactions directly from production systems, making it an effective tool for testing. Think of it as a more HTTP-focused version of tcpdump, with added optimizations tailored for traffic analysis.

Commands for Recording Traffic

At its core, GoReplay uses a straightforward input-output system to define where traffic is captured and where it’s sent. This makes it easy to start recording live data.

For a basic setup that captures traffic and prints it to your terminal:

sudo ./gor --input-raw :8000 --output-stdout

Want to send that captured traffic to a staging environment instead? Use:

sudo ./gor --input-raw :8000 --output-http http://staging.env

For example, Videology - a digital advertising platform - used this method to stream traffic from their production load balancers to various QA environments. This enabled them to run soak tests on real data while comparing performance metrics across environments.

Filtering and Limiting Traffic

Not all captured traffic is useful for testing, so GoReplay comes with plenty of options to filter and limit traffic. These features let you focus on the requests that matter most, ensuring your testing environment stays efficient.

Filter Type	Purpose	Example Usage
URL Path	Target specific endpoints	--input-raw-track-params api_version
HTTP Method	Filter by request type	--http-allow-method GET
Request Size	Control payload size	--http-max-size 1mb
Rate Limiting	Manage traffic volume	--input-raw-limit 10%

“GoReplay’s filtering capabilities allow us to stream a small slice of traffic from production load balancers, making it an invaluable tool for realistic testing scenarios”, says the engineering team at Videology.

Need to store your captured traffic for later processing or analysis? GoReplay has you covered with several output options:

• Stream directly to ElasticSearch for data analysis.
• Use Kafka for scalable, distributed traffic processing.
• Save traffic to Amazon S3 for long-term storage.

When handling sensitive data, it’s critical to exclude confidential information like headers or request parameters. GoReplay makes this easy by letting you disable such fields:

sudo ./gor --input-raw :8000 --output-http "http://staging.env" --http-disable-headers "Authorization,X-API-Key"

If you’re dealing with advanced setups, such as microservices that use binary protocols like Thrift or ProtocolBuffers, the Pro version extends support for these protocols, increasing its flexibility in complex architectures.

Replaying Traffic in Testing Environments

Once you’ve recorded production traffic, GoReplay makes it simple to replay that data in your testing environments. This allows teams to test new features and infrastructure changes using realistic usage patterns.

Setting Up Replay Endpoints

Configuring replay endpoints is crucial to ensuring traffic is accurately reproduced. Here’s a basic example to replay traffic to a staging environment:

sudo gor --input-file ./requests.gor --output-http http://staging.example.com

If you’re working across multiple environments, GoReplay supports replaying to multiple endpoints at once. This is especially useful for comparing how applications perform in different setups:

Replay Scenario	Command Example	Use Case
Single Endpoint	--output-http http://staging.env	Testing a single setup
Multiple Endpoints	--output-http "http://staging1.env|http://staging2.env"	Comparing versions
Rate-Limited Replay	--output-http "http://staging.env --rate-limit 10"	Simulating controlled load

Advanced Replay Options

GoReplay also offers powerful customization options for advanced replay needs. You can adjust traffic to better match your test requirements, especially for environments with specific data handling rules. For example, to rewrite headers or filter URLs:

sudo gor --input-file requests.gor \
    --output-http "http://staging.env" \
    --http-rewrite-header "Host: staging.example.com" \
    --http-allow-url ^/api/.*

“GoReplay’s filtering capabilities allow us to stream a small slice of traffic from production load balancers. It’s been a game-changer for catching performance issues early by comparing metrics across environments”, shares the engineering team at Videology.

For secure and accurate replay, GoReplay allows you to:

• Remove sensitive headers or cookies.
• Update authentication tokens to match staging or testing setups.
• Retain original client IPs to simulate actual user patterns.
• Modify request bodies to align with test environment needs.

When combined with CI/CD pipelines, traffic replay can be fully automated, ensuring every deployment is rigorously tested with real traffic before reaching production.

Adding GoReplay to Testing Processes

Incorporating GoReplay into your testing workflows can significantly improve deployment reliability by using actual production traffic patterns. Unlike synthetic tests, these patterns reflect real user behavior, offering better validation.

Shadow Testing

With GoReplay, shadow testing becomes straightforward. It lets teams duplicate live traffic and send it to staging environments without affecting production. Since it listens directly on the network interface, there’s no need to make changes to your production setup.

gor --input-tcp :80 --output-http "http://staging.example.com" --http-allow-url ^/api/.* --http-disallow-url ^/admin/.*

This configuration enables:

• Testing new features against realistic user behavior
• Validating infrastructure updates in a staging environment
• Spotting potential issues before they make it to production

Testing Aspect	Without GoReplay	With GoReplay Shadow Testing
Traffic Pattern	Simulated data	Real production patterns
Data Coverage	Limited scenarios	Diverse user flows
Issue Detection	Basic validations	Production-like testing

Load Testing

GoReplay shifts the way load testing is done by replacing artificial loads with real recorded traffic. You can play back these user patterns at different scales to find performance bottlenecks.

gor --input-file recorded_traffic.gor --output-http "http://staging.example.com" --rate 200%

“By implementing GoReplay in our testing pipeline, we identified performance bottlenecks that synthetic load testing missed. The ability to replay real user patterns helped us optimize our infrastructure more effectively”, says the engineering team at Speedscale, demonstrating just how impactful GoReplay can be.

For effective load testing, consider these tips:

• Start by running recorded traffic at its original rate (100%)
• Slowly increase the load to pinpoint the system’s breaking point
• Keep an eye on system metrics during replay sessions
• Use filtering to test specific API endpoints or user paths in detail

When combined with CI/CD pipelines, GoReplay allows you to automate both shadow and load tests. This ensures all deployments are tested under realistic conditions, making it easier to catch issues early and meet performance expectations.

Improving and Fixing GoReplay Setup

Getting the best out of GoReplay means nailing down its setup and tweaking key settings. Over time, teams using it have identified practical ways to fine-tune performance and build more efficient testing environments.

Tuning Performance

A key step in boosting performance is adjusting the worker configuration. Instead of relying on just one worker thread, spreading tasks across multiple workers can prevent bottlenecks and improve how much traffic the system can handle. To enable this, you can use the following example:

gor --input-raw :8080 --output-http "http://staging.example.com" --workers 4

Configuration Area	Effect	Suggested Setting
Worker Count	Balances Load	2–4 workers for medium traffic, 4–8 for heavy loads
Request Timeout	Improves Response Handling	5–10 seconds for standard APIs
Buffer Size	Manages Memory Usage	64MB for typical use, 128MB for heavy traffic

If your setup needs to deal with redirects, make sure you set timeouts to avoid piling up requests:

gor --input-raw :8080 --output-http "http://staging.example.com" --http-timeout 5

Security Tips

When it comes to security, protecting sensitive data during traffic replay is crucial. Thankfully, GoReplay comes with built-in tools for handling such challenges.

“After implementing GoReplay’s TLS configuration and data filtering, Videology successfully streamed production traffic to QA environments while maintaining strict security protocols. This enabled comprehensive soak testing without compromising sensitive user data”, shared the Videology engineering team.

To securely handle traffic, use TLS for encrypted input, as shown below:

gor --input-raw :443 --output-http "https://staging.example.com" --input-raw-tls-cert "/path/to/cert.pem" --input-raw-tls-key "/path/to/key.pem"

You can further safeguard sensitive data by setting up filters to block or replace sensitive information:

gor --input-raw :8080 --output-http "http://staging.example.com" --http-set-header "Authorization: FILTERED" --http-disallow-url "/admin/.*"

For teams needing even more advanced security options, GoReplay Pro adds premium features such as cloud storage integration and TCP session replication. These tools bring better testing capabilities while adhering to strict security standards.

Conclusion

GoReplay stands out as a practical tool for building test environments by capturing and replaying traffic. Its real-world value has been highlighted through successful use cases, such as Videology’s implementation for extensive soak testing across multiple QA setups.

The ability to manage HTTP traffic coupled with features for security and performance optimization makes it an effective choice for teams that need solid testing solutions. By fine-tuning worker configurations, timeout settings, and security approaches, teams can replicate production traffic while safeguarding data and maintaining system performance.

“GoReplay enabled us to stream production traffic to QA environments while maintaining strict security protocols, which was crucial for our comprehensive soak testing approach”, shared the Videology engineering team, demonstrating the tool’s utility in enterprise-level testing.

Testing Capability	Primary Benefit	Real-World Impact
Traffic Shadowing	No Production Risks	Safely tests live traffic patterns
Load Testing	Validates Performance	Detects bottlenecks before deployment
Security Features	Protects Data	Ensures TLS support and sensitive data filtering

For teams working on HTTP-based applications, GoReplay provides an easy way to set up end-to-end testing environments. While alternatives like Speedscale may better suit Kubernetes-specific needs, GoReplay’s ease of use and effectiveness for general HTTP testing makes it a valuable tool. With both open-source and Pro versions available, organizations can choose the right functionality to match their testing goals.