This post is the second of a three part series on using the Twitter API. If you missed part one, you can read it here.

Authenticating With Twitter via OAuth

Birdcage uses a Yii extension called Yii Twitter by Will Wharton, which makes use of the open-source PHP OAuth Twitter library by Abraham Williams.

I place the extension in the Yii tree under /app/protected/extensions/yiitwitteroauth. In Yii, you configure extension properties in the main.php configuration file like so:

    // application components
	'components'=>array(
	  'twitter' => array(
    				'class' => 'ext.yiitwitteroauth.YiiTwitter',
    				'consumer_key' => '',
    				'consumer_secret' => '',
    				'callback' => '',
    			),

Normally, I'd load these settings from my Yii .ini file, but to make the Birdcage setup simpler, I'm configuring the application keys from the UserSettings model. I've extended YiiTwitter.php to load the default user's application keys during initialization:

    public function init() {
	  // load twitter app keys from UserSetting table
    $result = UserSetting::model()->loadPrimarySettings();
    $this->consumer_key = $result['twitter_key'];
    $this->consumer_secret = $result['twitter_secret'];
    $this->callback = $result['twitter_url'];
		$this->registerScripts();
		parent::init();	
	}	

Once you've installed and configured the application settings, you'll need to visit the Accounts menu and click Add Your Twitter Account. 

When you click on the Twitter icon, it executes the Connect method of the Birdcage Twitter controller:

  public function actionConnect()
       {
        unset(Yii::app()->session['account_id']);
        Yii::app()->session['account_id']=$_GET['id'];
       $twitter = Yii::app()->twitter->getTwitter();  
       $request_token = $twitter->getRequestToken();
       //set some session info
       Yii::app()->session['oauth_token'] = $token =$request_token['oauth_token'];
       Yii::app()->session['oauth_token_secret'] = $request_token['oauth_token_secret'];

          if ($twitter->http_code == 200) {
              //get twitter connect url
              $url = $twitter->getAuthorizeURL($token);
              //send them              
              Yii::app()->request->redirect($url);
          }else{
              //error here
              $this->redirect(Yii::app()->homeUrl);
          }
      }

This will take you back to Twitter via OAuth to authenticate your Twitter account:

Once you've logged in, Twitter will ask you to authorize the Birdcage application:

Twitter will return the browser to your callback URL, our Twitter Controller Callback method. It will save your Twitter user OAuth token and secret in the account table:

  public function actionCallback() {
    /* If the oauth_token is old redirect to the connect page. */
            if (isset($_REQUEST['oauth_token']) && Yii::app()->session['oauth_token'] !== $_REQUEST['oauth_token']) {
                Yii::app()->session['oauth_status'] = 'oldtoken';
            }
/* Create TwitteroAuth object with app key/secret and token key/secret from default phase */
            $twitter = Yii::app()->twitter->getTwitterTokened(Yii::app()->session['oauth_token'], Yii::app()->session['oauth_token_secret']);   
            /* Request access tokens from twitter */
            $access_token = $twitter->getAccessToken($_REQUEST['oauth_verifier']);
      /* Save the access tokens. Normally these would be saved in a database for future use. */            
            Yii::app()->session['access_token'] = $access_token;
            $account = Account::model()->findByAttributes(array('user_id'=>Yii::app()->user->id,'id'=>Yii::app()->session['account_id']));
            $account['oauth_token'] = $access_token['oauth_token'];
            $account['oauth_token_secret'] = $access_token['oauth_token_secret'];
$account->save();
            
            /* Remove no longer needed request tokens */
            unset(Yii::app()->session['oauth_token']);
            unset(Yii::app()->session['oauth_token_secret']);

            if (200 == $twitter->http_code) {
          /* The user has been verified and the access tokens can be saved for future use */
                Yii::app()->session['status'] = 'verified';
                $this->redirect(array('account/admin'));

            } else {
                /* Save HTTP status for error dialog on connnect page.*/
                //header('Location: /clearsessions.php');
                $this->redirect(Yii::app()->homeUrl);
            } 
       }

Now, Birdcage is ready to begin making requests for Twitter data via the API on behalf of your user account.

As you'll see ahead, a simple call with the tokens allows access to the API:

$twitter = Yii::app()->twitter->getTwitterTokened($account['oauth_token'], $account['oauth_token_secret']);

Processing Tweets in the Background

For part two of our tutorial, we're using the Twitter REST API. Part three will delve into the real-time, always-on Streaming API:

Retrieving Twitter Timelines

Twitter timelines are an ever-expanding stack of tweets, so monitoring activity is a bit more complicated than your average REST API. You can learn more about the unique problem Twitter timelines present here. Essentially, as you're trying to read the timeline history, more tweets are being added all the time:

Twitter provides a relatively simple way to manage this. You can follow the code that performs this in Birdcage's Tweet model, getRecentTweets().

First, we look up the highest (most recent) tweet_id in our database, and return an incremented value:

  public function getLastTweet($account_id) {
    // get highest tweet_it where account_id = $account_id
    $criteria=new CDbCriteria;
    $criteria->select='max(tweet_id) AS max_tweet_id';
    $criteria->condition="account_id = ".$account_id;
    $row = Tweet::model()->find($criteria);
    if ($row['max_tweet_id'] ==0)
      return 1;
    else
      return $row['max_tweet_id']+1;
  }

Then, we request some number (e.g. 100) of tweets since the highest previously processed one. The Twitter API recognizes the since_id as a pointer to the place in the timeline you wish to start retrieving from. It will return all the tweets more recent than since_id. In the example below, we're querying the REST API statuses/home_timeline method. The home timeline is what a user sees on their main Twitter screen.

    $since_id = $this->getLastTweet($account->id);
    echo 'since: '.$since_id;lb();
    // retrieve tweets up until that last stored one
    $tweets= $twitter->get("statuses/home_timeline",array('count'=>100,'since_id'=>$since_id)); 
    if (count($tweets)==0) return false; // nothing returned

It's also important to check if we've been rate limited by Twitter. Each application-user request is allowed 180 requests to a user's home timeline per 15-minute window—but rate limits vary by activity, so you're programming mileage may vary.

For each tweet received, we call our Parse() method to process the data and store it in our various database tables. During the process, we track the oldest/lowest tweet_id that we've received from Twitter:

    foreach ($tweets as $i) {
      if ($low_id==0)
        $low_id = intval($i->id_str);
      else
        $low_id = min(intval($i->id_str),$low_id);
      Tweet::model()->parse($account->id,$i);
    }

The parse method adds the referenced Twitter user information and then the tweet itself. There's more detail in the Parse.php model.

  public function parse($account_id,$tweet) {
      // add user
      $tu = TwitterUser::model()->add($tweet->user);
      // add tweet
      $tweet_obj = $this->add($account_id,$tweet);

Then, we continue to request blocks of tweets using the lowest ID from the last request as a max_id parameter which we send to Twitter. We make these subsequent requests using the since_id of the tweet we began with and the max_id from the last oldest tweet we retrieved.

    // retrieve next block until our code limit reached
    while ($count_tweets <= $limit) {
      lb(2);
      $max_id = $low_id-1;
      $tweets= $twitter->get("statuses/home_timeline",array('count'=>100,'max_id'=>$max_id,'since_id'=>$since_id));
      if (count($tweets)==0) break;
      if ($this->isRateLimited($tweets)) return false;
      echo 'count'.count($tweets);lb();
      $count_tweets+=count($tweets);
      foreach ($tweets as $i) {
        $low_id = min(intval($i->id_str),$low_id);
        Tweet::model()->parse($account->id,$i);
      }              
    }

So, for example, as newer tweets come in, we don't see them—because Twitter is only sending us tweets since the initial highest tweet_id (since_id) from our database. We'll have to come back later to get newer tweets, which are higher than our initial since_id.

It's important to note that we do not receive an infinite number of older tweets. Twitter returns to us only the number of tweets we request that are older than the previous low ID or (max_id in our next call).

Once you get used to the model and nomenclature, it's quite simple.

While there is a Fetch menu command which will run this operation, we also configure a cron job to call our DaemonController method every five minutes:

# To define the time you can provide concrete values for
# minute (m), hour (h), day of month (dom), month (mon),
# and day of week (dow) or use '*' in these fields (for 'any').# 
# Notice that tasks will be started based on the cron's system
# daemon's notion of time and timezones.
# 
# For example, you can run a backup of all your user accounts
# at 5 a.m every week with:
# 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/
# 
# m h  dom mon dow   command
*/5 * * * * wget -O /dev/null http://birdcage.yourdomain.com/daemon/index

This in turn calls our getStreams method which performs the operations described above (note, the streams' functionality will be described in part three of this series):

    public function actionIndex() {
	  // if not using twitter streams, we'll process tweets by REST API
	  if (!Yii::app()->params['twitter_stream']) {
	    Tweet::model()->getStreams();	    
	  } else {
	    Stream::model()->process();
	  }
  }

The end result looks something like this:

One time I did run into some Twitter API reliability problems. You can check the status of the Twitter API services here.

Posting a Tweet

Posting tweets to your Twitter account is actually quite straightforward. We just need to make use of the statuses/update REST method. It takes a bit more work to perform accurate character counts. 

Twitter resolves all URLs into http://t.co shortcuts, so all URLs are counted as 20 characters. I needed JavaScript that would count characters and adjust for any URL by 20 characters regardless of the length of a URL. I settled on a combination of jQuery and JavaScript solutions, which I'll detail below.

I chose to create a model specifically for composing tweets called Status.php. This made it easier to work with Yii to generate forms for posting to the API. 

When you click on Compose in the Birdcage menu, it will take you to the Compose method of the StatusController:

    public function actionCompose($id=0)
	{	  
	  if (!UserSetting::model()->checkConfiguration(Yii::app()->user->id)) {
      Yii::app()->user->setFlash('warning','Please configure your Twitter settings.');
			$this->redirect(array('/usersetting/update'));					    
	  }
		
		$model=new Status;
		$model->account_id = $id;

		// Uncomment the following line if AJAX validation is needed
		// $this->performAjaxValidation($model);

		if(isset($_POST['Status']))
		{
			$model->attributes=$_POST['Status'];
			if ($model->account_id=='' or $model->account_id==0) {
		    Yii::app()->user->setFlash('no_account','You must select an account before tweeting.'); 			  
			  $this->redirect(array('status/compose'));
			}
      $model->created_at =new CDbExpression('NOW()'); 
      $model->modified_at =new CDbExpression('NOW()');

			if($model->save()) {
			  $account = Account::model()->findByPK($model->account_id);
			  $twitter = Yii::app()->twitter->getTwitterTokened($account['oauth_token'], $account['oauth_token_secret']);
        // retrieve tweets up until that last stored one
        $tweets= $twitter->post("statuses/update",array('status'=>$model->tweet_text)); 
				$this->redirect(array('view','id'=>$model->id));
        
			}
		}

		$this->render('compose',array(
			'model'=>$model,
		));
	}

This will load the HTML form for creating a Status item. Check out the _form.php in /app/protected/views/status/.

First, I'll load several jQuery and JavaScript libraries for character counting:

$baseUrl = Yii::app()->baseUrl; 
$cs = Yii::app()->getClientScript();
$cs->registerScriptFile($baseUrl.'/js/jquery.simplyCountable.js');
$cs->registerScriptFile($baseUrl.'/js/twitter-text.js');
$cs->registerScriptFile($baseUrl.'/js/twitter_count.js');

I used a combination of the jQuery simplyCountable plugin, twitter-text.js (a JavaScript-based Twitter text-processing script) and a script that did the heavy lifting of URL adjustments: twitter_count.js.

The following code then creates the remainder of the compose form and activates the character counting scripts:

<?php $form=$this->beginWidget('bootstrap.widgets.TbActiveForm',array(
    'id'=>'status-form',
	'enableAjaxValidation'=>false,
)); ?>

<?php 
  if(Yii::app()->user->hasFlash('no_account')
    ) {
  $this->widget('bootstrap.widgets.TbAlert', array(
      'alerts'=>array( // configurations per alert type
  	    'no_account'=>array('block'=>true, 'fade'=>true, 'closeText'=>'×'), 
      ),
  ));
}
?>

	<p class="help-block">Fields with <span class="required">*</span> are required.</p>

	<?php echo $form->errorSummary($model); ?>

  <?php 
    if ($model->account_id == 0 ) {
      echo CHtml::activeLabel($model,'account_id',array('label'=>'Tweet with Account:')); 
      $model->account_id = 1;
      echo CHtml::activeDropDownList($model,'account_id',Account::model()->getList(),array('empty'=>'Select an Account'));
    } else {
      echo CHtml::hiddenField('account_id',$model->account_id);
        }
  ?>

  <br />
	<?php 
	echo $form->textAreaRow($model,'tweet_text',array('id'=>'tweet_text','rows'=>6, 'cols'=>50, 'class'=>'span8'));
   ?>
   <p class="right">Remaining: <span id="counter2">0</span></p>

	<div class="form-actions">
		<?php $this->widget('bootstrap.widgets.TbButton', array(
			'buttonType'=>'submit',
			'type'=>'primary',
			'label'=>$model->isNewRecord ? 'Create' : 'Save',
		)); ?>
	</div>

<?php $this->endWidget(); ?>
<script type="text/javascript" charset="utf-8">
	$(document).ready(function()
	{
	  $('#tweet_text').simplyCountable({
	    counter: '#counter2',
      maxCount: 140,
      countDirection: 'down'
	  });
	});
</script>

The result looks like this:

When the tweet is saved, it executes this code in the StatusController—which posts the resulting tweet_text to Twitter via OAuth:

    		if($model->save()) {
			  $account = Account::model()->findByPK($model->account_id);
			  $twitter = Yii::app()->twitter->getTwitterTokened($account['oauth_token'], $account['oauth_token_secret']);
        // retrieve tweets up until that last stored one
        $tweets= $twitter->post("statuses/update",array('status'=>$model->tweet_text)); 
				$this->redirect(array('view','id'=>$model->id));        
			}

Next Steps

In this part of the series, we've reviewed how to authenticate with the Twitter API via OAuth, how to query for ranges of tweets in the user's timeline, and how to count characters and post tweets via the API. I hope you've found this useful.

Part three will cover using the Twitter Streaming API and the open source Phirehose streaming implementation.

Please post any comments, corrections, or additional ideas below. You can browse my other Tuts+ tutorials on my author page or follow me on Twitter @reifman.