Managing Data Sources in Legal Watch

This guide explains how to monitor, troubleshoot, and configure web scraping parameters for legal data sources. Completing this ensures your information feeds remain accurate, up-to-date, and free of persistent errors.

This workflow is primarily intended for legal engineers and system administrators responsible for maintaining data ingestion pipelines and resolving scraping failures in the platform.

Navigating the Dashboard

The Source Management dashboard provides a high-level overview of your data ingestion. At the top of the page, overview cards display counts for your total, active, pending, and errored sources.

You can quickly narrow down the visible sources using the filtering options available on this view:

Troubleshooting Failed Sources

When scraping tasks fail, use the dashboard to identify the root cause and re-run the job.

1

2

3

4

5

6

7

8

Editing and Previewing Sources

You can make targeted adjustments to a source's configuration or run test previews without affecting the live ingestion.

9

10

11

12

13

14

Q: Why did my source disappear from the list immediately after I clicked Trigger?

A: If you are actively filtering the dashboard to show only errored sources, re-triggering the source changes its status to active/pending. Because it is no longer in an error state, it drops out of your current filtered view until you clear the filters.

Q: Why can't I edit instructions for certain data sources?

A: Direct sources and sources using custom scrapers cannot be edited through the standard instruction interface. Direct sources automatically ingest a direct file or RSS feed, while custom scrapers require an engineer to manually update the underlying scraper code.

Q: Should I frequently edit the core setup of an approved source?

A: No. Generally, you should avoid editing the core configuration of an approved source. Modifying the fundamental content rules will require the source to undergo the legal expert review and approval process again.