About PolitiTweet


PolitiTweet is a Politiwatch initiative.

PolitiTweet monitors public (and primarily political) figures online, creating a stable and persistent database that can be referred to for news articles, citations, and other reference.

One primary advantage of PolitiTweet is that its database remains intact when, for example, a tweet is deleted (which Donald Trump does frequently).

When using PolitiTweet materials, please cite PolitiTweet prominently. The information provided by this website is licensed CC-BY, making it free for anyone to use provided that they cite PolitiTweet.


Database


PolitiTweet monitors 996 public figures. Its database contains over twenty million tweets, 292779 of which have been deleted from Twitter.


Advantages



Infrastructure


The back-end PolitiTweet database is managed by a Python daemon, is stored in JSON, and is stored in a filesystem. It is backed up weekly.

The PolitiTweet infrastructure has three parts: webserver, scanner, and streamer.

Its servers are located in New York City.


Webserver

The webserver handles the interface that you are interacting with right now. It handles processing and handling the database for end-user consumption. It is written in Python, Javascript, HTML, CSS, and uses JSON for data storage.


Scanner

The scanner does most of the work for PolitiTweet. It scans for new tweets four times every day, and automatically fills in any holes in the database. The scanner also detects any deleted tweets and deleted accounts.


Streamer

The PolitiTweet streamer is extremely fast. It pulls a stream of all the public figures' new tweets from Twitter, and archives the tweets into the database less than half a second after they are created. This allows PolitiTweet to have every* tweet, even those that only existed for one second before being deleted.


PolitiTweet Team


The PolitiTweet team is small. Contact the team here: securely or normally.



*Unfortunately, due to Twitter's API design, not every tweet by the public figures is available (Twitter has a date cut-off).