Code

Get All Twitter Posts of a Specific Hashtag in PHP

Note: The Twitter REST API v1 – which this script calls – is no longer active. I haven't had a chance to upgrade this article to API v1.1 yet.

I got a response to my article about pulling a Twitter feed into your site with PHP asking how to pull all tweets for a given #hashtag. Here, I have published an example of how to do so… It's not saved into a database, you'll have to do that yourself, but this should be enough to get you started. You can see the code below in action here, searching for the hashtag #yankees.

<?php
$pagetitle = 'Pull Twitter Hashtags';
require 'header.inc.php';

function getTweets($hash_tag) {

    $url = 'http://search.twitter.com/search.atom?q='.urlencode($hash_tag) ;
    echo '<p>Connecting to <strong>'.$url.'</strong>&hellip;</p>';
    $ch = curl_init($url);
    curl_setopt ($ch, CURLOPT_RETURNTRANSFER, TRUE);
    $xml = curl_exec ($ch);
    curl_close ($ch);

    //If you want to see the response from Twitter, uncomment this next part out:
    echo '<p>Response:</p>';
    echo '<pre>'.htmlspecialchars($xml).'</pre>';

    $affected = 0;
    $twelement = new SimpleXMLElement($xml);
    foreach ($twelement->entry as $entry) {
        $text = trim($entry->title);
        $author = trim($entry->author->name);
        $time = strtotime($entry->published);
        $id = $entry->id;
        echo '<p>Tweet from '.htmlspecialchars($author).': <strong>'.htmlspecialchars($text).'</strong>  <em>Posted '.date('n/j/y g:i a',$time).'</em></p>';
    }

    return true ;
}

getTweets('#yankees');

echo '<hr /><p>This page is an example for the article <a href="http://www.inkplant.com/code/get-twitter-posts-by-hashtag.php">Get All Twitter Posts of a Specific Hashtag in PHP</a>.</p>' ;

require 'footer.inc.php';
?>

There are a lot of options with the Twitter Search method. You can find out more by checking out their documentation.



42 Comments

June 1, 2011, 5:23 am
Hi Robert,

Sorry to be an absolute pain and ask you so many questions but I'm a complete newbie when it comes to writing PHP Code.

I have managed to merge the code code for pulling hashtags (above) with your code to pull a feed and write to a database but I keep getting a DB Error when I run the script although the script does write ONE record to my new empty database.

I've pasted my code below and would welcome any hints as to where I have gone wrong if you have the time to have a look at it please.

–- Start Paste –-

require_once 'db-functions.inc.php' ; //custom database functions

function saveTweets($hash_tag) {
$link = dbConnect('db user','db password','localhost','db name');

if (!$hash_tag) { echo "Error: No hash tag declared.n"; return false; }

$row = dbGetRow("SELECT `id` FROM `doncaster` WHERE `author`='$author' ORDER BY `id` DESC LIMIT 1");
$last_id = $row['id'];

$url = 'http://search.twitter.com/search.atom?q='.urlencode($hash_tag);

if ($last_id) { $url .= "&since_id=$last_id" ; }
$ch = curl_init($url);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, TRUE);
$xml = curl_exec ($ch);
curl_close ($ch);

$affected = 0;
$twelement = new SimpleXMLElement($xml);
foreach ($twelement->entry as $entry) {
$text = trim($entry->title);
$author = trim($entry->author->name);
$time = strtotime($entry->published);
$id = $entry->id;

dbQuery("INSERT INTO `doncaster` (`id`,`author`,`time`,`text`,`hidden`) VALUES ('$id','$author','$time','$text','n')");
$affected = $affected + dbAffectedRows();
}

return "".number_format($affected)." new tweets from $hash_tag saved.n" ;
}
echo saveTweets('#doncaster');

–- Finish Paste –-

Thank you in advance for any guidance you may be able to offer.

Michael :o)
Robert James Reese
June 1, 2011, 10:07 am
Hi Michael,

No worries. It's cool to know that someone is actually reading these things. :-)

What was the database error you got? I have a hunch that there was a single quote in one of the values you were trying to insert. It's important for that reason (and more importantly for security) to always escape the strings before inserting them. The PHP function for this is mysql_real_escape_string() but that's really long so I shortened it to dbEscape() in my database functions that I see you're using. So, try this:

dbQuery("INSERT INTO `doncaster` (`id`,`author`,`time`,`text`,`hidden`) VALUES ('".dbEscape($id)."','".dbEscape($author)."','$time','".dbEscape($text)."','n')");

Does that work?
June 1, 2011, 10:23 am
Hi Robert,

I've just updated the line that you've included above but still getting the same thing.

The script is currently installed on my 'test bed' - You can access it at: http://rover.doncaster.tk/rover3.php

The database error simply reads: "Error: Database error!"

Thank you so much for your assistance with this it's very much appreciated :-)
Robert James Reese
June 1, 2011, 10:33 am
Hey,

Two things to try to track it down:

Add "global $debug;" to your dbError function (I've updated it here to reflect that.)

And then, at the top of rover3.php add "$debug = true;" That should result in the actual database error being returned. When you get this ironed out, you'll want to remove that so that actual MySQL errors are never shown to your users (again for security reasons).
June 1, 2011, 10:45 am
Hey that's a really neat little command :OD

OK I've updated and the error message showing reads:
"Error: Database error!

Query: INSERT INTO `doncaster` (`id`,`author`,`time`,`text`,`hidden`) VALUES ('tag:search.twitter.com,2005:75926803759378432','andrewisaacs (Andrew Isaacs)','1306937335','RT @DMWmagazine: OK people. The all important announcement… who won the signed Rovers shirt?! Find out here on Friday! #doncasterisgreat #drfc','n')

Error: Duplicate entry '0' for key 1"
Robert James Reese
June 1, 2011, 10:52 am
Ah, okay... In the first post I did, the ID being returned was an integer, here it's not. So, the column type needs to change. Run this (only once) and then try again:

dbQuery("ALTER TABLE `doncaster` CHANGE `id` `id` VARCHAR( 255 ) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL");

After you run that once, delete it or comment it out.

After doing that, you should also update your query to this so that duplicates are ignored instead of throwing an error: dbQuery("INSERT IGNORE INTO `doncaster` (`id`,`author`,`time`,`text`,`hidden`) VALUES ('".dbEscape($id)."','".dbEscape($author)."','$time','".dbEscape($text)."','n')");

In the original post, the API call should not have returned duplicates, but in this new call, it might. The IGNORE tells MySQL not to throw an error when that happens.
June 1, 2011, 11:03 am
WOO HOO!! Thank you so much for helping me out - I'm dead chuffed as this is my very first PHP Script that I've attempted to assemble myself :D

OK on checking my DB I now have new posts with ID's that look like this:
"tag:search.twitter.com,2005:75927564367040513"

and Authors that look like this:
"MODEmagazineuk (MODE Magazine)"
Robert James Reese
June 1, 2011, 11:04 am
Nice work! Glad I could help.
Michael
June 1, 2011, 11:10 am
Thank you :o)

Just a couple of small things which would be good to know for the future:

1/ If I wanted to pull from more than one hash tag what would I need to add to my script to do so?

2/ Is there a way I can clean up the ID's so as they are just the numeric part or do I need to leave the test string on them?
Robert James Reese
June 1, 2011, 11:20 am
1.) Since you have it as a function already, you just call it again for each hashtag. For example:

echo saveTweets('#doncaster');
echo saveTweets('#yankees');

2.) Yes, you could, but I'd just leave them alone for simplicity's sake. Look into preg_replace() if you decide to. Here's an example:

$id = preg_replace('/[^0-9]/','',$id);
Muhammad Haseeb
August 10, 2011, 3:18 am
how do i get only my latest posting from twitter
Rob Abbott
September 3, 2011, 4:48 pm
How can I pull the tweeter's profile image, as well as turn the posts into links back to the original tweet? Thanks in advance!
November 30, 2011, 12:44 am
I am also trying to create a similar page to Michael. I am getting completely different errors.

My database name is _tweets, and the table name is twitter.

My code looks like:

entry as $entry) {
$text = trim($entry->title);
$author = trim($entry->author->name);
$time = strtotime($entry->published);
$id = $entry->id;

dbQuery("INSERT IGNORE INTO `twitter` (`id`,`author`,`time`,`text`,`hidden`) VALUES ('".dbEscape($id)."','".dbEscape($author)."','$time','".dbEscape($text)."','n')");
$affected = $affected + dbAffectedRows();
}

return "".number_format($affected)." new tweets from $hash_tag saved.n" ;
}
echo saveTweets('#NP');

?>

-END PASTE-

The errors I'm receiving are:

Warning: mysql_connect() [function.mysql-connect]: Access denied for user 'user'@'localhost' (using password: YES) in /home/ombian/public_html/db-functions.inc.php on line 33

Warning: mysql_select_db(): supplied argument is not a valid MySQL-Link resource in /home/ombian/public_html/db-functions.inc.php on line 35
There was a problem connecting to our database.

-END PASTE-

I copy and pasted your db-functions page, and just saved it with the .inc.php on the end. I'm not sure if that's the correct way to do it.

I appreciate all your help, this is an awesome website.
November 30, 2011, 7:27 am
Hi Hunter,

Those errors sound like you simply have the wrong MySQL username/password combo. Make sure to update the $db array (at the top of db-functions.inc.php) with the appropriate info for your server.
November 30, 2011, 11:29 am
Yeah, sorry I realized that mistake as soon as I pressed the submit button haha.

Now I am getting the error:

Fatal error: Call to undefined function dbconnect() in /home/ombian/public_html/tweetsToDatabase.php on line 14

I don't see a dbConnect function in db-connect, is that a custom function Michael made?

Thanks again.
November 30, 2011, 4:28 pm
I see where you answered that question on the original page, sorry about that. Also it must be re-sending my comment when I refresh, didn't mean for that to happen.

I'll comment again (hopefully saying I had success!) when I take out that dbconnect line.
November 30, 2011, 7:21 pm
No worries. Hope you're able to figure it out. And I will look into preventing duplicate posts on refresh when I get a spare moment…
December 2, 2011, 11:09 am
Now I am getting the error:

Database error!

There was a problem getting information from our database.

That looks like it's coming from db-functions.php. Any thoughts?

Thanks again.
Robert James Reese
December 2, 2011, 2:29 pm
There's a MySQL error being generated. If you set $GLOBALS['db_debug'] = true at the top of your page, then the actual error text will print out.
December 2, 2011, 4:38 pm
It worked! The debug showed me a few other problems and I got it fixed.

You're awesome, thanks. I'll definitely recommend this site any change I get.
December 2, 2011, 5:04 pm
any chance I get.*
Robert James Reese
December 3, 2011, 11:23 am
Hey, glad I was able to help out. I often don't have time to respond to the comments (and feel bad about it) but you caught me in a lull between projects. Good luck with the rest of your site!
Keida
December 17, 2011, 6:03 am
Robert James,
Thank you for your tutorial, I have managed to get this working really well!

I have an additional question:

When I search for a specific hashtag, I would like the script not to save the entire tweet in one colum in mysql, but put the tweet (which would have to be presented by users by a strict format, let's say: '#hashtag number text') in different columns, which would contain 'number' and 'text'. Any guess how I can break up the tweet, or tell mysql how to break up the tweet?
Robert James Reese
December 17, 2011, 9:03 am
Hi Keida,

It's possible to split it within either MySQL or PHP, but I'd recommend splitting it in PHP, it's easier that way. Look up the strpos and substr functions on php.net.
Keida
December 17, 2011, 9:09 am
Robert James,

Thanks,

I found out now that php will be fine, exploding will works as well I guess
Gopal
January 25, 2012, 6:23 am
Hi Robert

My boss wants me to create a MY Sql database that can be used to retrieve specific twitter users or those using a specific hashtag. Please let me know how I can start working on it.
James
January 25, 2012, 9:09 pm
Hi Rob,

Very much a newbie to this - this page was very helpful. Thank you!

Does your above code work completely independently of the Twitter API?

Thanks,
James
February 6, 2012, 11:31 am
Hey man, is it possible to set a post count using this code? I can't seem to figure it out.

Cheers!
Alex
February 26, 2012, 6:07 am
Nice job posting this online.. it helped me out a lot!
David
March 2, 2012, 8:14 am
Great article. It helped me out.

And Sam, add &rpp=2 in the URL variable.
Fede
April 8, 2012, 1:01 am
Cool stuff man, thanks for sharing.

It looks like your code is only getting the latest tweets, how would I get a greater number of tweets, like from a week ago?

Thanks.
Phil
April 28, 2012, 9:48 pm
This is great, along with the other guide on user names I can store what I want.
I only have a limited number of cron jobs available on my server, is it possible to run the user name and hash tag scripts on the same page, given they share variables? (I guess the question is how to clear and reuse the variables between scripts.)
Alexander
July 22, 2012, 10:47 pm
Wonderful your code, thank you very much. Help me with something, as I get the georss: point, i mean, the coordinates for storage in the db? I tried this: $coords = trim($entry->twitter:geo->georss:point); but the error results is: Parse error: syntax error, unexpected ':' in… Some help?
DIANE
August 6, 2012, 3:18 pm
I'm REALLY new to all of this, but hope you can answer my questions. My cousin is in a twitter competition that is based on the number of tweets that contain #TeamMelissa. She has a competitor that is racking up tweets for herself (#TeamElena), 20,000 over the last week, but only a few hundred show in the twitter feed. Will utilizing this code help me to determine where the tweets are coming from?
August 31, 2012, 1:16 pm
Simple and it works. Thanks.
December 19, 2012, 2:36 am
Great Work many thanks for you effort and sharing……..
i have one question with you…
i need all #tag responds in the same page..
but currently every refresh i'm getting only 15 responds on page. how can i get the all responds from the starting, also i need to count the total counts of responds , thanks in advance, and once again thanks for your above codings
mj
December 30, 2012, 1:53 pm
Any idea how I could modify this code to store the tweets in a custom wordpress table? I created one called 'wp_twitter' and I added the cron job on the server, but nothing is storing in the table when I run the script. No errors are being reported though…
March 14, 2013, 6:15 am
Thanks for the search.twitter.com link, it was all I needed for creating my own plugin. I couln't find a single plugin that combines @usertweets and #hashtags into one feed. changing .atom to .json made it alot easier too.
Mervin
March 20, 2013, 8:28 am
Hi,

Any idea how i can just pull the twitter id and images to be displayed? instead of just a twitter feed?
maddy
March 25, 2013, 4:17 am
Sorry for asking you really dumb question. but i am new to php.

I want to use your program to find latest tweets for query that user enters in html textbox. Wen i simply copy pasted you code n run on apache server i got following error:

Warning: require(header.inc) [function.require]: failed to open stream: No such file or directory in C:webstweeter.php on line 3

Fatal error: require() [function.require]: Failed opening required 'header.inc' (include_path='.;C:phppear') in C:webstweeter.php on line 3

Plz help.Its urgent.

Thanks in advance.
April 12, 2013, 10:00 am
Hi,

Thanks for all the help..

Can someone get older #tagged tweets, like a month - 2months ago?

Please help
umair
July 9, 2013, 8:09 am
Can you please send me downloadable link for this code

Leave a Comment

Name
Email
Website
Comment
Name and email are required. Your email will not be published.

This post was published on May 31st, 2011 and updated on March 10th, 2014 by Robert James Reese in the following categories: PHP, Twitter.

Before using any of the code or other content in this post, you must read and agree to our Terms & Conditions.

Copyright © 2014, Ink Plant. All rights reserved.