Data collection
Some of my research uses automated collectors that gather published material, such as news articles, from public websites. If one of them has visited your site, this page explains who runs it, how it behaves, and how to stop it.
Academic, non-commercial research. Collected material is never republished, and the data is never sold or used for any commercial purpose.
Who runs them
The collectors are run by Colm A. Fox, Associate Professor of Political Science at Singapore Management University (faculty profile on the SMU website · personal website), for academic research.
How they behave
Every collector listed here:
- names itself in every request, with a link to its own page;
- reads robots.txt and follows it;
- collects slowly, one request at a time, and stops if your site signals that it is busy;
- never uses proxies, never disguises itself, and never gets around a paywall or login. It logs in only where a publisher has agreed to this.
Each collector follows the robots.txt rules written for its own name and for all robots (User-agent: *).
Collectors
Each collector has its own page explaining what it collects and why. Collectors that have finished stay listed here, with their dates.
| Collector | Project | Where | When | Status |
|---|---|---|---|---|
| SEAPEDbot | SEAPED, the Southeast Asia Protest Event Database | News sites in Indonesia | Since November 2026 | Test run, headlines only |
How to stop them
Email colmfox@smu.edu.sg and collection from your site will stop within 24 hours. Or block a collector in your robots.txt by its name; each collector’s page shows the lines to use.
Contact
Colm A. Fox, Associate Professor of Political Science, School of Social Sciences, Singapore Management University · Faculty profile on the SMU website · colmfox@smu.edu.sg
Page last updated 18 November 2026.