Adding a twitter feed on a high traffic website

The previous post from me regarding Twitter to place a twitter feed on your site is a fantastically quick way to display your tweets on a page, however it will only be successful on low traffic sites or if that feed is only on a single page of a site. If you had your feed in a sidebar for example that displayed on every page of the site and/or the site gets alot of high traffic, Twitters “Rate Limit” will come into play, only giving you 150 requests an hour. Depending on how your using this and how many people view your site, those requests can diminish very quickly.

So with that in mind I thought I’d give you a few more options, both a little more extensive than the option linked above, but very secure and will allow you to bypass the rate limit.

By using Twitters API and OAuth, you are able to authenticate the use of your account and reading from it. To do this, it does require an access token, which is generated from your account by creating an application (very similar to facebook apps to enable the same thing, however twitters is very simple compared). To do this, login to your twitter account at http://dev.twitter.com and in the drop down under your account when logged in, click on “My Applications”. Click “Create a new application” and you’ll be met with a very simple process of naming and describing the app, and supplying the URL that it will be used on. The callback URL is optional, for what we are using it for in this example, it is not required. Agree to their terms and create away. That easy. On the screen you are presented with, scroll to the bottom and click “Create my access token”. This is the information you will require.

Using the PHP method mentioned on twitter and the OAuth/Twitter library, this is then done all in PHP using the following code:

define('CONSUMER_KEY', '##YOUR_CONSUMER_KEY##');
define('CONSUMER_SECRET', '##YOUR_CONSUMER_SECRET##');
define('OAUTH_TOKEN', '##YOUR_OAUTH_TOKEN##');
define('OAUTH_TOKEN_SECRET', '##YOUR_OAUTH_TOKEN_SECRET##');
function twitterAPI() {
$connection = new TwitterOAuth(CONSUMER_KEY, CONSUMER_SECRET, OAUTH_TOKEN, OAUTH_TOKEN_SECRET);
return $connection;
}
$connection = twitterAPI();
$content = $connection->get("statuses/home_timeline");

$content will then contain your tweets information like the other example I posted up, which you can then display out however you need it to.

Alternatively if you, for some reason, don’t have access to the account you need to be able to create the access token and obtain the other details (say for example you wanted to display your favourite sportsmans tweets on your site), you can use Lupo Montero’s method of fetching tweets and displaying them (or his plugin that he has written) which is a more in-depth version of my method, but includes ajax caching to reduce the rate limit calls and also has the use of the “tweeted on” time/date.

Force Download large files with PHP

So you’ve been able to create zips on the fly with php but now you need to allow them to be downloaded. Lets assume that you don’t want to disclose the whereabouts on the server the files reside and what their filenames are but you want them to be downloaded by the masses. I hear you saying “surely thats impossible”, but you couldn’t be any further from the truth.

Using the header function and the content-type attribute can allow you to output a file in its own context. By default you could output to the page the following:

header("Content-Description: File Transfer");
header("Content-Disposition: attachment; filename=your_specified_filename.ext");
header("Content-Type: application/octet-stream");
header("Content-Transfer-Encoding: binary");
header('Expires: 0');
header('Cache-Control: must-revalidate');
header('Pragma: public');
$path = $_SERVER['DOCUMENT_ROOT']."path/to/file/filename.ext";
$fp = fopen($path, 'rb');
fpassthru($fp);

And that would output your file to the user with your specified filename for the file. Please note the “Content-Type” line. I have used “octet-stream” in this example which will cover the majority of files you would pass to be downloaded, if you were dynamically passing a number of different extensions. If you were only allowing PDF’s for example, you could just use:

header("Content-Type: application/pdf");

This would render all files passed to the page as a PDF. Obviously if you were to pass a JPG to that page, the PDF would be highly corrupt and would not run. A full list of MIME types you could use can be found on Wiki – MIME types.

The above on a general server will allow a file of about 40Mb to be downloaded before the server will timeout for security reasons. Recently I had a client with this problem. The zips I was creating for him was containing a random number of high res images which resulted in some zips being almost double the limit. Given that his clients still needed to be able to obtain these created files, a solution needed to be found. I COULD have just disclosed the full path of the files, would would have been very long, and messy, and with the way I was storing these files in the specific naming convention I had so that no zip was overwritten, the filename was very unreadable to the user.

After speaking to a very clued up server monkey on support for the client, he notified me of a header that allowed the server to do the processing of the file instead of the browser which removed the timeout issue. Unfortunately this setting is not normally turned on by default, but if you have a standalone or VPS server that your able to change settings on, this is THE perfect solution for you. Just add ONE more header line:

header('X-Sendfile: $path');

And suddenly you are passing files of any size to the user with no timeout troubles.

Now to set this up you can either talk to your hosting providers support team that should be able to help you out in enabling ‘mod_xsendfile’ or you can add some code to your .htaccess file:

<ifModule mod_xsendfile.c>
XSendFile On
XSendFilePath /home/$USER/public_html/folder/
</ifModule>

Again, depending on your server setup, this may or may not work. When in doubt, contact your support team.

CSS3 @font-face and sub domains

As many of you have probably noticed, I use two non-system fonts on this site for the headings. I build my sites initially using Chrome which showed the fonts working perfectly on both my main sites pages as well as the blog which is viewed via sub domain. Upon doing some cross-browser testing I quickly noticed that both Firefox and IE didn’t display the fonts on this blog site, even though the font were relative to the CSS file on the sub domain. Scratching my head for a bit, I found a little ‘.htaccess’ code helped this issue along:
<FilesMatch "\.(ttf|otf|woff)$">
<IfModule mod_headers.c>
Header set Access-Control-Allow-Origin "*"
</IfModule>
</FilesMatch>

Placing your twitter feed on your site

An earlier post described the very involved process of including a Facebook status list to your website. Now let’s check out how to get a twitter feed onto your page, and I promise you, it is no where near as time consuming as the Facebook feed!

Due to twitter being available to the public from that start, twitter don’t need to have extensive security when it comes to reading and returning a users feed. Because of this, the process of displaying your tweets on your site is only a few lines of code!

Ok, so lets start by creating an element for the data to go into, something like:
<div id="twitter_feed"><h3>Tweets</h3></div>

Great, the hard part is done ;) . Now to the twitter API and jQuery. JQuery’s getJSON function is perfect for this as twitter has the option to return the information needed in jSON format:
$.getJSON("http://twitter.com/statuses/user_timeline/ashrdurham.json?callback=?&count=3", function(data) {
$.each(data, function(i, object) {
$("#twitter_feed").append(object.text+"<br /><br />");
});
});

Complicated isn’t it? In the example above, the getJSON function is calling on twitter via URL and getting the data from a username, passing the variable count. For more details on this see the Twitter Developer Docs. Once the data is retrieved, we loop through each tweet and append it to the specified element. Simple right?

Now, I hear the questions, “What about links in the tweet? How can we make it so that they are actually links?”. Well I’m glad you asked. By adding the following function to the bottom of your javascript file:
function detect_link(text){
return text.replace(/(href="|)?[A-Za-z]+:\/\/[A-Za-z0-9-_]+\.[A-Za-z0-9-_:%&\?\/.=]+/g, function($0, $1) {
return $1 ? $0 : $0.link($0);
});
}

And modifying the twitter call slightly:
$.getJSON("http://twitter.com/statuses/user_timeline/ashrdurham.json?callback=?&count=3", function(data) {
$.each(data, function(i, object) {
$("#twitter_feed").append(detect_link(object.text)+"<br /><br />");
});
});

will turn any links in a post to clickable version of them.

Enjoy!

When in doubt, jQuery is the answer

Its not a surprise to hear that jQuery is a powerful tool in a web developers arsenal of languages to have. More and more I turn to jQuery in moments of desperation when I find that what I need to do is either impossible in that situation using traditional means (ie adding an element or class to an element into automatically generated code that you have no control over) or to do something a designer has just thought of and thinks it would be cool to have on the page even though in a web developers mind, it’s not even something that would be contemplated. JQuery always comes to the party.

The amount of functions available make this library so easy to use, especially when there are direct equivalents to functions that exist in php, like trim. So far there hasn’t been a problem that couldn’t be solved with jQuery.

Examples of this is when your using a user-friendly system that allows you to enter, let’s say a form, into a page which links into the systems databases and functions. Because this system is user-friendly system for the non-technical folk out there, it doesn’t allow modification to the tags that are given. Now enter designer that wants the form to have a two column form with labels above each input field, validation with a note per field and the submit button in a format using CSS3 elements. The form you have to work with has no containers around each field, no classes or ids and the submit button is defaulted using an image to give the non-technical user some hope of having a form that doesn’t just look like any old form. In a world without jQuery, this situation ended well before the designer.

However we do live in a world with jQuery so we have a mountain of options we can use here. First of all, you can add classes to anything you need to (for examples sake, let’s just forget that in the CSS you could just use “input[name=email]” as a selector) by getting the name of the field and using that as the class:
$("input").each(function() {
$(this).addClass($(this).attr("name")+"_field");
});

Or add a container around the label and input field:
$("label").before('<div class="field_container">');
$("input").after('</div>');

Or convert that pesky image submit button back to a normal submit button:
$("input[type=image]").removeAttr('src').attr('type', 'submit').attr('value', 'Send');

All very simple pieces of jQuery that have a powerful effect on how you can change a form that you previously had no control over. So instead of throwing your hands in the air and saying it can’t be done, grab the jQuery library and find a solution. Anything can be done, just depends on how long you have to do it ;)

The Social Nightmare – Facebook Feed on a Website

Social networks have quickly become THE way to get your business name into the market. Word of mouth has always been the strongest means of advertisement in the world and social networking pushes that even further. For an online presence, the social scene is seen as the fastest way to get ahead, especially Facebook.

How often do you now see a sign in a cafe or restaurant to like their Facebook page for special deals? Do they really want to give you special deals that bad? Of course not, they just want they elusive “person count” to go up even more so they can say, “we are the best restaurant in town, just look at how many people think so!”.

With that aside, we web developers are always asked to include social networking into sites, some more than others, and occasionally you’ll get a client that wants to get real smart and show their Facebook wall on their site. Now if said client had said to me they wanted their Twitter or Tumblr account’s wall to be displayed somewhere on their website, I wouldn’t even hesitate in saying no problem. But with Facebook’s recent security increases on accounts, it has become increasing more difficult to port the information across. Not to mention that every time they do make a considerable update, you need to redo the way you are porting the information across.

Just recently I had a situation with another developer at work that had to get a Facebook and Twitter feed of their wall onto the website. Twitter was done in a matter of minutes. Facebook on the other hand, given that it had been few and far between the last time I had done the port, I began to think was also going to be a simple’ish’ task. How wrong was I!? I have found that there just isn’t a spelt out way to do this process. I had to piece together different peoples methods of doing this and decode Facebook’s API explanation as to what to do. SO this is what this post is for. Spelling out EXACTLY what needs to happen to get a Facebook feed onto a website.

I am basing this off getting a Facebook Page wall onto a site. What you need to do this is:

  • A Facebook Account
  • A Facebook Page with the above Facebook account as an Admin to the page
  • A website

Ok, so not the hardest list in the world right?

Step 1) Create a Facebook App
Log into the Facebook account that is an admin of the Facebook page you want to port the wall from and go to https://developers.facebook.com/apps. Click on the “Create New App” button, fill out the form (Name and Namespace) and submit. Congratulations, you have just created a Facebook App. This can be used for a whole range of things to do with Facebook, not just for what we’re doing here. Facebook apps like the games and things are created this way. An option in the advanced settings of the app under “Migrations” needs to be disabled, “Remove offline_access permission”. This needs to be set to disabled to allow an extended access token allowance.

Step 2) Get your Access Code
I would have to say that this is the most undocumented part of the process. Facebook do have a great way to retrieve the dreaded Access Token for your application however very well hidden. Down the left hand side of the screen you are on for your newly created app, you should see a link for “Graph API Explorer”. Open this in a new tab/window. You should be shown your own details for your account when loaded. You can have a little bit of fun here. Facebook have been very smart in how they allow permissions to gain access to your pages for external use at the same time as giving maximum security to your data. Their API uses the “graph” extension of Facebook to make all of these calls. Basically you can call any Facebook ID or User ID into the graph and get all of the publicly accessible information. What we want to do here is get the information from a page, so go ahead and replace the ID number that has been automatically added to the graph URL with your pages link ID (this is either the numbered ID of your page or the chosen URL you have). In my case, it is “durhamnetau” and hit submit. You will be shown some information about your page.

Now obviously this isn’t enough to get your status updates from as you can see. At the top of the page, you should see a dropdown with the label “Application”. Your newly created app should already be selected. If not, select it now and click on the “Get Access Token” button. You will be given a popup window to set the permissions you want to give the app for your page. This is up to you what you want to allow the app to have control over your page. What we need most for this example is under the Extended Permissions tab. Check the “Publish Stream” option which should be in the first column, 5 down. And also the “offline_access” which is in the third column, 3 down. Go ahead and click on “Get Access Token”, you will be asked to “Login” to approve of this action in another window, then to allow the app to post on your behalf. Allowing this will close the window and populate the Access Token box with a very long string. You now have the access token you need to continue.

Step 3) Add some jQuery to your website to do the heavy lifting
There are a few people that have written code to pull the information you need out and return it to you in a way that you can format any way you would like. I have found a nice lightweight script from http://www.prettyklicks.com using the getJSON function in jQuery. I have had to do some tweeks to bring the script up to latest standards and to use the access token. Create a javascript file on your server, lets call it fbwall.js, grab the following code and paste it in:


(function($){
$.fn.fbstatus = function(options) {
set = jQuery.extend({
username: '', // [string] required
count: 5, // [integer] how many status updates to display?
access_token: '', // [string] Access Token
loading_text: null // [string] optional loading text, displayed while tweets load
}, options);
function fbstatus_link(text){
return text.replace(/(href="|)?[A-Za-z]+:\/\/[A-Za-z0-9-_]+\.[A-Za-z0-9-_:%&\?\/.=]+/g, function($0, $1) {
return $1 ? $0 : $0.link($0);
});
}
//Set Url of JSON data from the facebook graph api. make sure callback is set with a '?' to overcome the cross domain problems with JSON
var url = "https://graph.facebook.com/"+set.username+"/feed?access_token="+set.access_token+"&limit="+set.count+"&callback=?";
$(this).each(function(i, widget){
var loading = $('<p class="loading">'+set.loading_text+'</p>');
var theObject = $(this);
if (set.loading_text) $(widget).append(loading);
//Use jQuery getJSON method to fetch the data from the url and then create our unordered list with the relevant data.
$.getJSON(url,function(json){
var html = "<ul>";
//loop through and within data array's retrieve the message variable.
$.each(json.data,function(i,fb){
if (fb.message) {
html += "<li>" + fbstatus_link(fb.message) + "</li>";
}
});
html += "</ul>";
//A little animation once fetched
theObject.animate({opacity:0}, 500, function(){
theObject.children(".loading").remove();
theObject.append(html);
});
theObject.animate({opacity:1}, 500);
});
});
};
})(jQuery);

Basically what the above code is doing is using the graph API call on Facebook with the username, access token and the amount of items you want to pull and outputting them into a tidy list for you. Add either your page’s ID or name (mine is durhamnetau) as the username, put in your access token, and chose the amount of feeds you want to pull and save your script. You can also add some loading text if you would like.

Step 5) jQuery code to output the data to a specific area on your site
Now that everything is in place, add the following code to your page, whether it be in a functions js file or in the header or footer of your page directly:


<script type="text/javascript" src="fbwall.js"></script>
<script type="text/javascript">
$(document).ready(function() {
$("#fb_wall").fbstatus();
});
</script>

Alternatively you can change any of the settings at this point aswell:


<script type="text/javascript" src="fbwall.js"></script>
<script type="text/javascript">
$(document).ready(function() {
$("#fb_wall").fbstatus({
username: 'durhamnetau',
loading_text: 'Loading Feeds...'
});
});
</script>

Add a div or span to your page where you would like the feed to go with the id ‘fb_wall’. Save it all and refresh your page and watch your Facebook feeds rush onto your sites pages!

So that was pretty simple right? Sure, when its documented like that, its a piece of cake….now. You can see mine in action on the main page of this site: http://durham.net.au

Zip it up with PHP

There is a lot of things that PHP can do, so many functions it has which you could never know them all off the top of your head. File management and creation is quite an interesting side of the PHP library.

I’m currently doing a project at work that requires a lot of heavy lifting to create, upload and manage files across a great deal of users. One part of it requires the user to dynamically choose a set of files to either download or send via email, straight from the site. Now, depending on the user, there could be quite a number of files that they want sent at one given time, large files at that. You could create links to each file and list each one in an email, however I’d hate to be the other end of that. Logical way to move these files is to zip them up into one neat little package… If only we could just right click and compress them right?

There are quite a few libraries out there for PHP that do this sort of stuff reasonably well, but as stated in a previous post, I’m not one to want to just take a library and call a function to do what I want. Easy to do, yes. Really quick way to get want I want, yes. Any idea how it happens, fat chance. PHP has a raw class called ZipArchive.

ZipArchive class can do everything you can do on a PC or a Mac. Unzip and existing zip file and store the extracted files on the server, zip up files existing on the server or create files on the fly and add them to the zip file. Take the following code for example:


<?php
$zip = new ZipArchive;
$arch = $zip->open('test.zip', ZipArchive::CREATE);
if ($arch === TRUE) {
$zip->addFromString('new_file.txt', 'dynamically created content can go in here...!');
$zip->addFile('new_name.txt', 'current_name.txt');
$zip->close();
echo 'Zip file created successfully';
} else {
echo 'Zip file failed to create.';
}
?>

The code above starts the ZipArchive instance, opens to a new zip to create (if the file already exists in this location, this will return an error. Using “ZipArchive::OVERWRITE” will delete the file with the name given in the open function)’ creates and adds a new text file with specified content, adds an already existing file and renaming the file to another name and finally closing (finalizing) the zip ready to be distributed.

This is all well and good if you want to have set files added to the zip every time, but I would think a dynamic option would be better, either dynamically selecting a different folder name of files and folders or building a file-manager-like system that allows you to create an array of files at add. For my said project, I have users logging in and uploading files into a created folder for that user with sub folders for each “subject”. When multiple files are requested for a particular subject, a zip is created with the files contained in that folder.

To do this, I have created a function to extend on the ZipArchive class called addDirectory as there is no function in the ZipArchive set to add entire directories. Here is the function:


<?php
class Zip extends ZipArchive {
public function addDirectory($dir, $start_dir, $rel_path, $include = array(), $exclude_dirs = array()) {
foreach(glob($dir . '/*') as $file) {
$file = str_replace('//', '/', $file);
if(is_dir($file)) {
$path = explode('/', $file);
end($path);
if (!in_array(current($path), $exclude_dirs)) {
$this->addDirectory($file, $start_dir, $rel_path, $include);
}
} else {
$path = explode('/', $file);
$parts = pathinfo($file);
if (!empty($include) && in_array($parts['basename'], $include) && $parts['extension'] <> 'zip') {
$begin = false;
$new_path = '';
for($p=0;$p if ($path[$p] == $start_dir) {
$begin = TRUE;
} elseif ($begin) {
$new_path .= $path[$p]."/";
}
}
$new_path = substr($new_path, 0, -1);
$this->addFile($file, $rel_path.$new_path);
} elseif (empty($include) && $parts['extension'] <> 'zip') {
$begin = false;
$new_path = '';
for($p=0;$p if ($path[$p] == $start_dir) {
$begin = TRUE;
} elseif ($begin) {
$new_path .= $path[$p]."/";
}
}
$new_path = substr($new_path, 0, -1);
$this->addFile($file, $rel_path.$new_path);
}
}
}
}
}
?>

An explanation? So you pass the root path of the folder of files (and folders) you want zipped as $dir. $start_dir is the folder name that you want to begin within the zip. For example if you have /home/abc/public_html/files/username/photos/may-12-2012-01.jpg as a folder structure and you want to have the zip to have the folders /photos/may-12-2012-01.jpg then $start_dir would be username. $rel_path gives you the ability to allow a parent folder to your zip, so by setting $rel_path to Company Name – Files the file structure when unzipped would be /Company Name – Files/photos/may-12-2012-01.jpg.

$include can be used if you only want specific files from folders and $exclude_dirs can exclude certain folders from being included in the zip. Now with all of that explained, the use:


<?php
$zip = New Zip;
if (!is_dir("/home/abc/public_html/files/username/zips/")) { mkdir("/home/abc/public_html/files/username/zips/", 0777); }
if ($zip->open("/home/abc/public_html/files/username/zips/photos.zip", ZipArchive::OVERWRITE) === TRUE) {
$zip->addDirectory('/home/abc/public_html/files/username/photos/', 'username', 'Company Name - Files/');
$zip->close();
} else {
die("Error creating ZIP");
}
?>

Once running this you should find a new folder ‘zips’ under the ‘username’ folder containing a file ‘photos.zip’. Downloading this file and unzipping the zip file, you should then have a new folder on your machine called ‘Company Name – Files’ which contains the folder ‘photos’ that contains all the files within the photos folder on your server.

Now you can go ahead and zip up as many files and folders as you want! Enjoy!

The art of CSS – Pure CSS Arrows

In the last year or so I have concentrated on the front end side of my skills and really developed my skills in the use of CSS and jQuery. CSS in particular has dazzled me in just how much you can do with it.

Chris Coyier from CSS Tricks is one amazing man that has shown me the light in a lot of different ways to do things that I never thought possible with CSS. He has created a page on his blog displaying raw shapes that have been created with pure CSS.

Being able to create such shapes and icons with CSS benefits a number of things like page load and performance. Yes, IE is not the best browser to be checking some of these things out on due to the lack of support in earlier versions, but as time goes on, naturally the need to worry about these earlier versions will be no more.

Further more to these great CSS shapes, the use of the pseudo-elements in CSS are a very powerful tool, in particular the “:before” and “:after” pseudos. The arrows in the blog menu on this page are from using the :before pseudo together with the code to make a triangle. This can be attached to any element you require it on, list styles I would say to be the most common. For example:

The HTML
<ul class="arrow_list">
<li>List item 1</li>
<li>List item 2</li>
</ul>

Together with the CSS:
.arrow_list {
margin-left: 0;
}
.arrow_list li {
position: relative;
list-style: none;
padding-left: 15px;
}
.arrow_list li:before {
position: absolute;
left: 0;
top: 50%;
width: 0;
height: 0;
content: '';
display: block;
border: 5px solid transparent;
border-left: 5px solid #000;
margin-top: -5px;
}

Will produce:

  • List item 1
  • List item 2

Birth of the Insight

Welcome to the Birth of the Insight. My name is Ashley Durham, a LAMP Web Developer in Sydney Australia and this blog is being started for a few reasons:

  • To make the world aware of a reasonably talented web developer in a corner of the globe,
  • Chat about the latest releases, ideas, subjects going around the net in regards to web development and design,
  • Share some of the code that I write for projects that may just help others with theirs.

At this point your probably thinking, “Ah, just another one of these sites…”, to which I respond with… Maybe so, however my goal here to give yet another resource option to the vast amount out there. If you are a web developer then you can relate with me here in that you spend a good amount of time on google researching ways to gain an effect for a job, or figure out how to use some sort of function in a specific way. This is what this is all about! A real developer, doing these sorts of things day in day out, sharing that with you here. Think of it as a summary of information that nine times out of ten you will be looking for.

The beginning of this blog will probably be slow going giving that I am only one man and holding a full time development job does consume a lot of your time, even worse when you are passionate about what you do and how you do it. So many times I have searched for an hour or two on the best ways to go about something to either finally find what i need buried away on page 20 of a forum post or 56 comments down on a blog post (which by the way will not happen here as I will be updating the main post with any supported comment data that comes through), find 3 close ways to do it and need to piece together a few to give me what I need or use that as inspiration and create my own or not find anything and need to spend more time figuring it all out.

Ideally I would love to hit every hurdle I find head on by figuring it all out myself and store the solution in the memory bank for next time, unfortunately its not easy to do when you have to meet deadlines on projects. Due to this I often find myself crawling the net for the solution instead of re-creating the wheel. Downside to this is that you barely learn how it fixed your problem or how it actually works. Plugins are a major sucker for this in site builds, jQuery being the biggest player. I love jQuery and take every opportunity I get to build my own functions for sites over just finding a plugin, purely to better myself in the code but as usual the dreaded deadline kicks in and I need a quick solution to come in on time. Don’t get me wrong, there are some awesome plugins out there that you just can’t go past. I will be sharing the ones I like with you in time.

I will rave on more about all of these things in good time, I do have to give you a reason to came back you know ;) . In the meantime I again welcome you to the blog and do hope you bookmark this site and return regularly. I will certainly do my best to give you plenty of reading to do and stay on top of the latest in the industry.