Experts Round Table Network

Hello everybody. This is my first post, so let me introduce myself:
Iacopo, 38, Milan, Italy.
Glad of having found this community!

As stated in subject, my problem is making the user aware of server being producing a file:


Nothing new, so far. My problem is that the file is filled up with tons of data, so the web page reloads relatively quickly, while the download window pops up much later. I know it is "thinking", but the final user will be disoriented and will click here and there uselessly, flodding the server.

Is there a way to make the server lock the client while the file is processed, maybe showing an animation in the meantime?
I was thinking about some header that tells the server to listen to the file creation job and retrieve the "ondone" event, but in the HTTP reference I could'nt find anything useful.
Would somebody help me?
Thx!

Hello Iacopo, and welcome.

You want something like the downloads on sourceforge?

You could open a page that says "Your download will start automatically, please wait".
Then, initiate your file download from that page.

You can do that using javascript... something like this:
    document.onload = function() { window.location = "download-file.php"; }

Or you might be able to do it using an iframe (to be tested)... something like this:
    <iframe src="download-file.php">

Where "download-file.php" is the script that generates your huge file.

Uh, I didn't even figure out such a simple solution! Too easy, thank you for forcing me think!

Well, actually I was thinking about showing some animation like a progress bar, a wheeling clock or so on until the download pops up but still remains the problem of controlling it runtime on the client from the server.

Mumble! <scratching head>

About the animation,

You can have a cycling progress bar (that doesn't show the actual percentage) until you get a javascript confirmation that the page is done being generated - for example, you can do this using frames:

Or something similar with the onLoad event of the target page.

The animation could be anything that moves illustrating an ongoing progress with no specific end. For example, I created this graphic for a site of mine, used to the tune of:

Generating, one moment please...
(http://www.hispamedios.com/im/load_bar2.gif)

If you like it, feel free to snatch it for your use. I'm sure there are also plenty of those little animations available online.

BTW, Welcome!

yes, right, and if ever you want a real progress bar, then welcom to "the" club. A lot of people had/have/will have this problem :D

personally, I "solved" it in different ways. Quick tip :
- an hidden iframe with a refresh to get status in DB or via xmlhttprequest (= AJAX) so that the "real" frame/window can update the progress bar
- synchronizing on file size (via fs commands)
- synchronozing on actual progress as exposed by the working process - it writes the % in a rewritten file - and I read that file periodically. When at 100% the progres sbar disappears
- synch'ing via DB (this is a varinat of the above)

some people use auto-generating html code. It writes javscript that writes html (DOM etc)... fun but tricky


lots of ideas do work.

I'm quite sure I never found out the "real" solution, so stay tuned, people like grand Schtroumph and esopo probably have it :D

> <frameset rows="1%,*" onLoad="PageDone();">
I don't think there is a standard way for browsers to handle such code.
When should the "onload" be triggered?
When the "save as" dialog opens or when the download has finished?

I agree with VRG.
Your "file generator" needs to write the current progress somewhere (DB, flat file, session) where it's available to other processes.
Then you need to provide a method to fetch the current status trough HTTP.
Your page will need to query the server each time you want to update the progress.
I don't think there is a solution that is more "real" than that... unless you use some plugin that allows the server to push data to the client (flash? java applet?).

Esopo, VGR, GrandSchtroumpf, great advices!
File generator writes "start" in $_SESSION['progress'] before actually starting, then, just before die() command it changes value to "done"; a loop in a php page loaded by the same iframe that shows the animation polls the session variable until it finds "done" and the game IS done!
<jumping here and there>
Henchmen at work, thank you all!
Iac

> a loop in a php page loaded by the same iframe that shows the animation polls the session variable until it finds "done"

A loop in PHP!  That's an interesting idea for a progress bar...  I'll try to come up with some proof of concept.

Maybe AJAX to run the check on the session variable?

http://www.expertsrt.com/articles/VGR/ajax-demythified.html

the problem of polling the variable in the same process/thread/iframe/task than the script which displays the progress bar is that it may put the CPU at 100% because I'm not confident at all on th ecapability of php implemantations on win/*nix to handle such a situation. The keywords are "test and set", "sleep" and a good implementation would produce a thread/task that would consume VERY FEW CPU cycles to check the variable, see it's not modified (or readable) and sleep until next time. A bad implementation would be at 10% CPU, cycling over itself in an infinite loop...

personally, I would use two iframes and two php scripts and too bad if one of the two (it's a php loop of course) is at 100% cou, the other will not beblocked.

non-blocking stuff doesn't work properly on windaube, also (see fsockread() or the example on "non blocking sockets read" for instance)

The PHP sleep() method works fine on llinux, not on windoze.
I have a working proof of concept here locally.  I'll upload it soon.

If we only want to determine when the download starts, we just need to call session_start() in both scripts:

The "file-generator script" calls session_start(), then builds the document.
The "file-status script" sleeps for a while then calls session_start(), then echoes "DONE".

The fact that the "status script" sleeps ensures the generator script calls session_start() first.
It works on my local system.  It looks like we cannot have 2 concurrent processes that use the same session... The result is that the "status script" waits for the "generator script" to exit and then echoes "DONE".

Here we go:

PROGRESS-BAR
http://www.arsalis.com/lab/progress-bar/

DONE STATUS
http://www.arsalis.com/lab/done-status/

Some parts of that script rely on the flush() method...  this means it won't work with all server configurations.  However, it should work on all browsers that support iframes.

All that is done with single HTTP requests ONLY.
No javascript, no AJAX, no HTTP refresh, no META refresh.
Only 3 HTTP requests are issued: one for the main page, one for the generator iframe and one for the progress/status iframe.

Hello
As for "DONE STATUS" script, the problem is that I already have a session, users are logged in to use the application. So my script cannot rely on that entity.
PROGRESS-BAR seems to answer my purpose, it's more or less what I thought about:

iframeA.php
<?php
/**
 * Opens file allocating it
 */
$handle = fopen($filename, 'w+');

while($row = mysql_fetch_row($result))
{
   fwrite($handle, $row['value']);
}
/**
 * Free file
 */
fclose($handle);
?>

iframeB.php
<?php
do
{
   /**
   * Do not overload CPU (Thx VGR and Linux server)
   */
   sleep(2);

   /**
   * Tries opening file, not allowed until iframeA.php is allocating it -> loop
   */
   $handle = @fopen($filename, 'w+b');
} while(!isset($handle)); // Exits when succeeds opening file

/**
 * Free file, no need to use it but to know it's available
 */
fclose($handle);

/**
 * Send file
 */
header("Content-type: text/html");
header("Content-Disposition: attachment; filename=\"generated-file.html\"");
?>

Gonna try it as soon as navvy's free!

a) In my two examples, there are 3 different HTTP requests:  main-page, file-generator, file-status.

I would definitely create the file and send it in the same HTTP request.  It makes more sense.

If you don't need to keep the generated file on disk for later use, you should not write it to the disk.  Simply echo the file content.  You can still use a file lock to pass the done-status to the other HTTP request without having to write your content to the file.


b) The session side-effect (using the standard session module) is that 2 different HTTP requests that use "session_start()" cannot execute simultaneously.

If your application uses sessions to identify your users, you will surely need to call "session_start()" in the request that sends the file (to check user privileges).  You probably also want to call "session_start()" in the "main-page" request to display some menus, a log-off link or other stuff.  Your status request should not be required to call it since that's not sensitive data.


c) If all 3 HTTP requests use "session_start()", then we might have a problem to force the order of execution of the "file-generator" and the "file-status" when "main-page" takes a very long time to return.  It depends on how the queuing works on the server.

Scenario:
1. "main-page" starts session and echoes the frames which generate the 2 other HTTP requests.
2. "file-generator" and "file-status" both reach the call to "session_start()" and get locked.
3. "main-page" returns and unlocks the 2 other requests.
4. ???  Which locked process should have the priority?  Is it FIFO, LIFO or Random?  Based on time at which the HTTP request arrived or the time at which the session_start was called?  Does that depend on the PHP version or on the platform?

Other than that risk, using the technique shown in my "DONE-STATUS" example should work fine in your particular case.
You might prefer to use a safer technique where your status request does not start a session.

Here is an interesting discussion on the session locking mechanism:
http://ajaxian.com/archives/troubles-with-asynchronous-ajax-requests-and-php-sessions


d) Make sure you use some max-iterations counter in your loop that sleeps.  The maximum execution time for PHP scripts is 30 secs by default.  Sleeping time is not counted as execution time on Linux.  For instance, if one loop iteration takes 0.01 sec to execute then sleeps for 2 seconds, the script will die after 6000 seconds if the loop never exits.

point (b) is no longer a problem if you provide a session_id in the session_start() call or using session_name() [if i'm not mistaken]

this is necessary if you use the same server for multiple sites and don't run the risk of cross-site session hijacking.

for point (c) on onwards, I must state that I encountered on many occasions problems with php sessions being too slow to establish session variable values , in the case iof a frame refreshing an other one for instance. I used to add a delay using sleep() - quite distrubing ; i shou duse a handler <> files, ie db, and LOCK explicitly but I'm too lazy

> point (b) is no longer a problem if you provide a session_id in the session_start() call or using session_name()
Thanks for that tip.  I still have a lot to learn in PHP.
VGR, did you take a look at my experiments?  What do you think about them?

no time, seau d'riz. Really.
newt week-end perhaps