The attached ebook, Files in Drupal, has been expanded and updated to account for changes in Drupal 6.
— Update, May 10th, 2008.
You'll find in the pdf document attached to this posting “10 things you ought to know” about file download in Drupal, from a developer’s perspective. This small guide provides sample code, recipes, concise yet complete explanations, tricks, and a thorough coverage of the module hook function file_download. The document is 6 11 16 17 pages long, it has been expanded and updated to account for changes in Drupal 5 and 6. I also added a 'funny' cover to the book. (My modest attempt at humour.) Typos are corrected pretty much on a daily basis... If you’ve downloaded the document before May 17th, 2008, please redownload it.
Summary of the “10 things you ought to know about file download in drupal” :
The problem : Using Drupal’s in-core upload module, and no “node access control” contributed module, anonymous users can either download all files attachments, or none of them. The trick : in the case where anonymous users are forbidden access to uploaded files, automatically generate a message to inform them that one or more files are attached to a post. Involves : modifications to node.tpl.php and the use of Drupal’s function format_plural(). This section provides sample code safe to copy from the pdf to your template file in your favorite text editor. download permission
The problem : Drupal’s core upload module does not keep track of file downloads, even when the download method is set to private. The trick : install the contributed module download_count. This module writes a descriptive message in your logs whenever a person or robot is attempting to download an attached file. The module also keeps a record of total download hits, and last download time, to display on a dedicated page and in nodes (to which these files are attached). In your tracking of file downloads, you can chose to disregard file attachments with particular extensions. download statistics
This third point explains when, why and how to use the hook function file_download. Using this function, we can :
This section provides two code examples : one in which we allow the download/display of an image that sits outside the web root, and an other in which we allow the download of any file through an “open or save to disk” prompt. module development
This point describes 5 Drupal functions to use when dealing with file download :
The section explains when to use these functions and what to expect from them. It also provides sample code. file path
This section explains how to use private download — and why we should use the ../privateFolder notation when specifying our File System Path. File System Path for private download
Challenge : we need to change when and how the attachment table is displayed. The solution : we use the themeable function theme_upload_attachments($files) to print the table in node.tpl.php — hence printing the value returned by theme('upload_attachments', $files) — or we override that function in template.php. This section provides sample code, and it reminds us of the difference between the variables $teaser and $node->teaser in the template file node.tpl.php : $teaser is a boolean, TRUE if we are to display the teaser as content, while $node->teaser is the content of that teaser. attachment table
What we should know about uploading two files with the same name when using the Drupal core upload module. filename versus filepath
This section explains what hot linking is, and presents 3 different methods to try and prevent it. However, I only recommend the first method, and explain why the others are flawed (these others use the value $_SERVER['HTTP-REFERER']). leaching
The problem : when clean URLs is enabled, we cannot use relative paths to files and images in our html markup. This problem has been discussed on Drupal.org. There are at least 4 solutions to this problem :
This section provides a recipe to scan directories for particular files using the Drupal function file_scan_directory. It provides sample code on how to pick a random image from a folder and all its subfolders. scanning the file system path folder
Attachment | Size | Hits | Last download |
---|---|---|---|
Files-in-Drupal.pdf | 2.67 MB | 914 | 10 years 42 weeks ago |
Comments
I modified the PFD document
... and updated my summary of it.
A quick question...
Hi Caroline, and thanks for offering your lovely ebook to the world. I am using your 'warning message' code in a website, but for some reason the line
Is causing every node to break and I get an empty white page. As soon as I remove that line and just have
etc etc, everything goes back to normal. It seems very strange to me because the format_plural function is nothing unusual. Would you have any idea why this might occur? I can submit the entire node.tpl.php if you like...
Hi there...
There's a space between format_ and plural.
Looks like my code is not safe to copy from the PDF file to your own template, so I will fix that.The PDF is fine. When I select the entire code snippet in my PDF file and paste it into my node.tpl.php file (in Aptana) as is, no space appears there.Fixed!
Thanks for that. It's always the ones right under your nose you can't see.
Always
So true.
The white space
That white space certainly wasn't obvious to me at first. It took me a while to see it. I had time to open Aptana Studio (a long-drag-ass launch) before my eyes caught it,
.If it helps other people...
I'm using Smultron on a mac... When I copy and paste from Preview to Smultron I get spaces AND line breaks I now notice. But it might be more to do with Preview. When I get a chance I'll install Adobe Reader and try it out with Textwrangler as well.
files-in-drupal
With this module, would I be able to attach both private and public files to a page? How does the system know that these files are private and these other ones are public?
Or maybe it doesn't work like that?
Thanks!
Tom
what module?
What module?
Thank you
for accept my registration. I was looking something useful about Drupal's file handling and I found it here. Great article!
WOW Thank you!
You are very welcome.
Thanks a lot!
Thanks a lot for accepting my registration. :)
Altough forcing to register seemed a bit strange at first, it's the perfect way to keep the attention to your site.. and for good reasons. Your PDF about file handling rocks! And your implementation of drupal on this site is remarkably good as well.... pfew :)
Thank you
Thank you so much for this informative article. You are awesome.
Another choice on point 9
You may also want to use the URL replace filter.
I've outlined its use for the specific "image" attachment case on a page explaining how it replaces the img_relocator module , but it is more general that that, usable for all types of URLs.
Hello FGM
There are 2 problems with your module, the URL Replace Filter.
1. There isn't (yet) a Drupal 6 release for it.
2. It doesn't create URL of the form http://, hence links in content displayed on another web site will be broken. Don't forget feeds.
Maybe I am wrong here, and if I am, your project page needs some work:
The module Path Filter has been around. It has been recommended to me in the Drupal forums. It seems to have a good user base, and has been ported to Drupal 6.
Have a splendid day!
URL replace filter
Hi Caroline,
Thanks for your comments. Note that this is David Lesieur's module, not mine.
The problem you mentioned had indeed been in the issue queue for url_replace_filter for a few days when I suggested this module. The patch is currently awaiting RTBC by David.
Regarding the D6 version, someone suggested that the module might be replaced by flexifilter and, although I haven't verified things, I looks like this might be a good option. I don't know yet if David intends to do a D6 port or send users to flexifilter.
Path filter seems to be more limited, allowing only the "internal:" prefix to be redefined, but it confirms there has been several answers to the need for "solid" URLs.
What's a "PFD" document?
@Caroline
>I modified the PFD document.
What's a PFD? a pretty fucking document?
...
PFD stands for pretty fucked document. You were close.
PFD
Silly me! I always thought PFD meant Personal Flotation Device (lifejacket).
Funny!
I think I might have found some issue with Drupal 6.
I submitted an issue on Drupal.org (I am chill35 there).
http://drupal.org/node/257716
Basically it goes like this: as long as a user has the permission to 'view uploaded files', he can download just any file on the site, even these files that are attached to unpublished nodes, or to nodes the user has no access to. It's no biggie, but it's odd that this 'node access' check was dropped from Drupal 5 to Drupal 6.
The code for the function upload_file_download in Drupal 6 is:
<?php
/**
* Implementation of hook_file_download().
*/
function upload_file_download($file) {
if (!user_access('view uploaded files')) {
return -1;
}
$file = file_create_path($file);
$result = db_query("SELECT f.* FROM {files} f INNER JOIN {upload} u ON f.fid = u.fid WHERE filepath = '%s'", $file);
if ($file = db_fetch_object($result)) {
return array(
'Content-Type: '. $file->filemime,
'Content-Length: '. $file->filesize,
);
}
}
?>
In my opinion, it should be:
<?php
/**
* Implementation of hook_file_download().
*/
function upload_file_download($file) {
if (!user_access('view uploaded files')) {
return -1;
}
$file = file_create_path($file);
$result = db_query("SELECT f.* FROM {files} f INNER JOIN {upload} u ON f.fid = u.fid WHERE filepath = '%s'", $file);
if ($file = db_fetch_object($result)) {
$node = node_load($file->nid);
if (node_access('view', $node)) {
return array(
'Content-Type: ' . $file->filemime,
'Content-Length: ' . $file->filesize,
);
}
else {
return -1;
}
}
}
?>
Maybe the check was dropped...
... because there's something new in Drupal 6: files are associated (mainly) to users, instead of to nodes. Even though there's a nid in the {upload} table for each file uploaded. I read that the same file could be attached to many nodes... The long discussion I skimmed through is here: http://drupal.org/node/115267
Great writeup
Thanks for the writeup. Really appreciated!
Explain protection
Hello,
Thank you for the great explanation, it really gave me a lot of insight, it would be nice if you had a real world example which implements all that you say in the book.
Could you please let me know how you protect the download of the book and request registration, is it a module, or custom code?
Thanks
Custom code added to node.tpl.php.
The answer to your question is given in the ebook, point #1.
Thank you, thank you
So in your example you are just reprinting the attachment table without the links instead of printing the message.. Cool now I get it
No.
Instead of printing the message? On top of printing the message! The message and the attachment table are 2 different things. The message is generated by some php code in node.tpl.php. Now, if you want the attachment table without the links, that would be the work of the contributed module download_count. I thought you were talking only about the message that says 'blabla... login or register to download files'...
oO
>
o
page not found?
Hi,
Thanks for the book. It's really helpful. I'm running into a problem, though, that maybe you know the answer to. I'm trying to set up private files, so I moved my files dir to ../../private_files/mysite/ and changed the setting on the file system page. I have a page with a list of pdfs which I generate by doing a directory scan in a custom module. (The files have been manually uploaded.) It seems to be working the way it should: the files are all listed, and have urls like http://mysite.com/system/files/docs/myfile.pdf. But when you click on one of the links, you get Page Not Found.
I've cleared the cache, and I thought it might have been a permissions issue, but private_files and everything below is set to 777. (Does it actually need to be that open?) The File System admin page gives all kinds of dire warnings about changing the privacy setting after the site has been running, which I've done, but isn't at all specific about what the problems might be. I could imagine they'd be with paths in uploaded files, but I'm not dealing with that here. Any other ideas?
TIA!
P.S. I'm on D5 for this project.
P.P.S. Ooooh! Duh! The files have to exist in the database in order for drupal to know anything about their permissions. Hmm.... that's a drag. I really want to have sub-directories for these files, which means I really don't want to use drupal's file upload system.
Private upload
Check out this module: http://drupal.org/project/private_upload
Private upload
Yeah, I found that. (Thanks, though.) As far as I can tell, you get one level of hierarchy -- main and private, but you still don't get the ability to have a structure to your uploads. There's also http://drupal.org/project/filebrowser, which may do the trick, but I haven't had a chance to look at it yet.
Thanks for sharing!
Hi,
I arrived after reading some of your comments on Drupal.org.
Very usefull information, thank you!
Chears, Joep
Great documentation
Thanks for sharing your knowledge. Some parts of Drupal are not always easy to understand, your ebook helps a lot.
How will the files be found?
Assuming that one restricts nodes with uploaded files to authenticated users and also denies anonymous users the ability to view uploaded files, are search engines and bots still able to or likely to find and index any of the uploaded files? In other words will this strategy keep them safe from being indexed or does one still need to set the download method to private?
Private method
Very likely, yes. As they will be in a web-accessible folder. The file robots.txt can be used to tell search engines what to index and what not to index, hence you could add your /files folder to it, but not all search engines will listen.
One needs to put his downloadable files in a folder that is not web-accessible, and because of that one needs to use the private method.
Thanks for sharing!
(and for approving my registration!) the book was quite useful.
I have a quick question - on page 9 you recommend typing ../private as file system path.
Now that seems to do the following
public_html
---drupal
-------files
---private
i.e. it puts the private directory "two" steps above where the files directory should go (right?)
But if I want to go "three" steps above - since my drupal installation is within drupal directory which eats away a step - what should I put?
./../private?
../../private?
.../private? (I think I tried this and it doesn't seem to work)
I think the second one does the trick but is that the right way to go? I have absolutely zero intuition with file paths and would desperately try to avoid having to fix paths once I have many files online :D
also, in your opinion, what should I put as temporary directory? The default value is /tmp - should I just leave that?
thanks and btw the cover image for your book is adorable :)
Thanks for the approval Caroline - and great cover!
File downloads sure are tricky with Drupal when it comes to multi-layer memberships roles. I'd love to be able to show a file exists but deny permission to access/download it depending on roles - (like your site here), so I really appreciate you giving away this material.
Muchas Gracias Caroline!
Robert
hi!
Thanks so much for preparing these pdfs for everyone. It must have taken a lot of dedication and effort.
I'm trying to develop a drupal page that is able to manage files and control access. I was hoping that maybe I could get access to your pdf tutorial? I would greatly appreciate it.
Thanks,
Justin
Could really use this pdf
Hi,
I'd like to download this pdf and learn about files as I am having some issues that I think will help.
I tried to register for an account about 2 weeks ago but it never got activated.
Am I supposed to do something to get a new account activated?
Thanks and slight error in thing #10
Hi,
thanks for your work. I found your site while looking for a Drupal 6 random image module or code.
Even the little teaser snippet was enough to show me how to do it myself but I still wanted to register and see what else was on offer :)
I've just read through the pdf and found it very interesting, I'm sure I'll your site useful since I only started with php and Drupal 4 weeks ago; just got a job at a small software / web developer company coming from a c# desktop app background.
When checking to see how different our code was I found the only real difference was that I didn't know about array_rand and was doing it manualy with count() and mt_rand() so I've just changed my code which led me to notice that your example is slightly wrong.
array_rand returns a key not a value so you need to change it to:
$image = $images[array_rand($images)];
Once again thanks for your hard work and unselfishness, I'll be back to see what else
Why does it work
The following code works — and it was the source of my confusion. It is taken from the inspire module, who has been created and tested by me. The module inspire is used on this very site, and it works top notch.
Now, the question is, why does it work? The function file_scan_directory() returns an associative array, keyed on the filename, of objects with "path", "basename", and "name" properties. Here is why it works: $image is the full path to the file. Both functions basename() and image_get_info() can accept the full path to a file.
Now, I will go correct the code in the tutorial, and reupload it.
Thanks, Alan.
Simple when I think about it
OK,
First here is the code I originally used:
When I found out about arr_rand from your pdf I changed my code to:
My error is glaringly obvious now I stop to think about it - I left the array_values() around the call to file_scan_directory() which meant that array_rand() gave me back a number instead of the filename.
Of course the filename key works fine in your code but not in mine, Doh! Now I'll have to rewrite my code to be even more like yours.
Well we live and learn - and mainly it's to look at your the code more carefully :)
edit: btw I know that the second code doesn't work thats why I had to make the change I first posted about above.
But you were correct, Alan!
The tutorial is wrong. It is trying to access the $image as if it was an object, rather than a filename. I have exported the PDF file and will now update the attachment... But yeah, in the code, it useful (sometimes) to just use the full path to the file. And the full path to the files... are the keys of the array returned by the function file_scan_directory(). Interesting.
I am sorry. I edited my earlier reply. Quickly, but you were quicker than me. Probably because you got an email notification of my reply.
Oh yes
Right, I get it - in the code snippet you posted above you're not using $image as an object but in the pdf you are, same as I was in my code before I saw your reply.
Now we've sorted that out I think I'll change my code back to grabbing and using the object rather than calling basename() on the filename;
Cheers,
Alan.
Community-aware
That document has been downloaded 18439 times, and no one had reported this error to me yet. Most people are not very 'community-aware' for sure.
Thanks heaps - and a question re images in nodes
Hi Caroline,
Thanks heaps for the great document, it was a major help !
I also got a question regarding the display of uploaded images, I hope that's ok.
I'm currently developing custom modules and I use the private download method to restrict access to uploaded files to authenticated users only.
Referring to chapter 10 in your doc, I use file_create_url() to link to images in the File System Path which is outside of my web accessible folders. It creates an appropriate URL with the system/files part in it (with my subfolder structure in mind something like: http://www.mysite.com/mytest/system/files/images/events/example.jpg; 'mysite' being my Drupal folder).
However, the custom node only shows placeholders for the images, not the actual images. (with placeholders I mean the rectangles with the red 'x' in the corner). I use file_create_path() to determine the size of the images, and that works fine as the placeholders have the correct size (but no image).
Am I missing something?
Directory permissions of the File System Path are set to 777.
I don't know if it matters, but I 'manually' upload documents and images using file_save_upload(). Thus, the Drupal db does officially not know about these files, but I assumed that that shouldn't matter, right?
Thanks so much for your help!
Cheers from Brisbane,
Natascha
Oops... got it
After re-reading chapter 3 of your documentation, I (finally) realised that I need a hook file_download in my custom module. Works perfectly fine now, all images displayed! Thanks again for the doc! :-)
I don't know if it matters...
I am sorry I came in late to see your question. I am glad you figured it out. Yes, if you upload a file without using the upload module, and you do not record the uploaded file info in the upload module table, you will need to implement HOOK_file_download.
Thank you, Natascha, for your donation. It came in at a time of need.
file handling in drupal
Caroline, nice clear document, thank you.
It's interesting that most stuff I've read about drupal file handling is around explicit upload/download though most file handling I care about is putting images in text content. I assume, and so far it works for me, that putting HTML inline in the content - or using tinyMCE/IMCE - the info about file download is just as relevant to those links as to the use of separate attachment lists appended to nodes.
Something addressing the technique of embedding inline thumbnails with a mouse-over or one-click display of a pop-up full-size version, and how to protect both thumbnail and full sized image would be useful.
My particular application problem is the embedding inline of images (actually faxes of documents) where the thumbnail and full-size versions are access-controlled by role - with the file_download hook I should find that it's merely a SMOP.
If you know of specific references to info on stuff like this, it'd enhance your PDF if you had maybe a references bit at the end.
Oh, and as a relative novice at these things, wouldn't x/y/inspire1/... or .../Iminspired/ match:
if (strstr($relativepath, 'inspire'))? - maybe a delimiter or two in the needle would help?
Thanks for a good article.
...Ian.
protecting private files (drupal sense of Private)
More of an aside, in case anyone reading here has a similar problem:
You recommend placing the files subdirectory outside the "document root" (in Apache-speak) so that there's no way to submit a URL directly accessing the files, even if you could guess the name and file path. That's pretty secure.
Recently I had reason to want to give FTP access to upload 50+ images to a colleague but I didn't want to give carte blanche for that FTP account to write outside the document root.
So, while not quite as good as what you recommend as the only good way to do it, I used the technique used by MediaWiki's image-protection mechanism. While placing the /files subdirectory inside the document root I added an .htaccess file to the /files subdirectory with just the line
Deny from all
Barring mistakes or someone with access to your Linux account it's going to be pretty hard for a browser to get access.
...Ian.
can't download the pdf even when logged in??
Hi Caroline -
I can see the link to the ebook pdf in your article, and I am logged into the site, but the link is not clickable, so I can't download it... am I doing something wrong??
Under the non-clickable link, I am seeing the text "One file is attached to this posting. Login or register to download this file." But I am logged in...
Cheers,
Pete
Premium account
You need a premium account to download files and view premium content. The change was effective a few weeks ago. I will send further instructions to your email address. I will also modify that misleading text for the benefit of people who registered before the change. Thank you.
Done.
Done.
hmm
It's useless IMHO
Thanks
Thanks, your ebook was my best guide trough this topic. Was going mad setting this up for a client.
Keep up the hard work, and let me know if I can help with time/effort.
Gaby
Visitors downloading everything!
OK, so I've used your ebook (very nice thank you) to make sure users are logged in before downloading content from my teachers' resources sharing site. The problem is, people login and then download everything - rather than just the stuff they need. This eats bandwidth and thus costs money!
Any ideas about restricting the number of downloads a users can make, say within an hour or day. Unless you have any other ideas about tackling the problem.
Thanks :)
There probably is a Drupal module for this
You need to keep a record of who downloaded what, and save that to your Drupal database. There must be a Drupal module that restricts number of downloads per period of time. If you can't find such module, then you'll need to write your own. If you write your own module, you'll implement the hook file_download to block all download when the maximum number of downloads has been reached for a particular user. The hook is explained on page 5 of the Files in Drupal ebook, under “3/10 » For module developers : the HOOK_file_download”.
Great
Thnks
Bought your book but couldn't download
Ah, the irony :-)
I paid via paypal for your book and upon download I got a "something went wrong, we have been informed" page.
How can I now get the pdf of the Files in Drupal book?
Tell me about it.
I am using this service here, and it only works half of the time.
Sent by email :-(
File has been sent your way by email.
Apologies,
Caroline
Failed download
Havin the same issue, and have had no response to an email sent regarding this.
Can it be sorted, or should I have PayPal force a refund?
Hi there, I have bought your
Hi there, I have bought your ebook and have used it with Drupal 6. Any pointers getting file download tables, downloads shown in teasers and 'warning - must be logged in to download' messages on Drupal 7? Thanks.
Drupal 7
Drupal 7 has now been out for a while and I wonder if your ebook will be useful here, too, if not, I won't bother checking it out.
It would be great to have file downloads in drupal 7 as described above, but I think that all those additional modules are just not yet up to the version...
Am I right? Am I wrong?
Thanks
It's a great ebook, especially for those who have a keen interest in Drupal and want to explore it in greater depths.