Example application using twitter api
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Jeffery Yeary 673f8c3cfc add some annotations to the readme 6 months ago
app pathing issues in prod 6 months ago
bin initial working version, needs clean up 6 months ago
config fixing schema to match assignment, adding rspecs 6 months ago
db fixing schema to match assignment, adding rspecs 6 months ago
lib/services fixing schema to match assignment, adding rspecs 6 months ago
public rails_apps_composer: add README files 6 months ago
spec fixing schema to match assignment, adding rspecs 6 months ago
vendor rails_apps_composer: initial commit 6 months ago
.env.production.sample simplify docker 6 months ago
.gitignore initial working version, needs clean up 6 months ago
.rspec rails_apps_composer: testing framework 6 months ago
.ruby-gemset rails_apps_composer: create database 6 months ago
.ruby-version rails_apps_composer: create database 6 months ago
Dockerfile simplify docker 6 months ago
Gemfile fixing schema to match assignment, adding rspecs 6 months ago
Gemfile.lock fixing schema to match assignment, adding rspecs 6 months ago
README.md add some annotations to the readme 6 months ago
Rakefile rails_apps_composer: initial commit 6 months ago
config.ru rails_apps_composer: initial commit 6 months ago
docker-compose.yml simplify docker 6 months ago
package.json rails_apps_composer: initial commit 6 months ago

README.md

Original Assignment:

Create an application that allows an external user to click a button, authenticate with Twitter, and in the background, pull down and store their last 30 days of Twitter activity in a database.

Please store the activity with the following data structure (hash keys are meant to indicate DB columns):

{
  "id"          => "3059251-1017096774220206080",
  "author_id"   => "760811609967079424",
  "author_name" => "Skyler Tonkin",
  "message"     => "example message",
  "message_id"  => "message id",
  "provider"    => "twitter",
  "type"        => "tweet",
  "link"        => "https://twitter.com/skyler_tonkin/status/1017096774220206080",
  "sent"        => true,
  "posted_at"   => 1531329742.0,
  "created_at"  => 1531330594.3471084,
 }

posted_at - Float indicating the time the content was created on Twitter

created_at - Float indicating the time the content was created in the application

sent - Boolean indicating if this person sent the message or not

The other fields should be self-explanatory.

Explanation:

I have a demo of the app up at https://civil.chat

When you log in your session is populated with oauth token and secret. I then queue up the TwitterSyncJob which basically just loops thru your user’s timeline 200 tweets at a time until we run out of tweets or until we hit a message older than 30days. I then take each tweet queue it up to the IngestTweetJob which inserts it into the database.

After login you are put back on the homepage and you can click the refresh button to see the messages as they stream in.

Technology I used:

  • Standard Rails 5 template with foundation grid
  • I am using twitter-omniauth gem to do 3-way oauth flow
  • I am using the twitter gem to make the calls to the twitter api

Things I would spend more time on:

  • The UI is basically non-existent. It is just a grid of results. Not dynamic (you have to refresh the page)
  • I don’t handle twitter api throttling / communication errors, they would just enter sidekiq retry
  • Testing is a little rough and could be cleaned up and expanded to cover more
  • The TwitterSyncJob is doing a lot of work. I might fan out the work more (maybe each page of results as its own job?)
  • Sidekiq is configured in one giant queue, might add additional queues and some priorities
  • The docker image I whipped up is really large. I would redo it to use alpine as a base
  • Every log in starts the ingestion process again, there is some protection to prevent duplicate messages but I would redo it to start the ingestion from the last message processed