From M.Arndt at science-computing.de Tue Oct 10 10:44:30 2006 From: M.Arndt at science-computing.de (Michael Arndt) Date: Tue, 10 Oct 2006 16:44:30 +0200 Subject: [xml-qstat-d] xml-qstat error / refresh / customize Message-ID: <20061010144430.GA18569@winter.science-computing.de> Hello * thx for making xml-qstat available ! 3 Questions: -how a automatic refesh of the page is done best ? -we have only 4 "Queues" but they result in many ( 160) Instances ( number of queues x hosts ) Is there a way to create a view in between summary and detail view, so that only queues running jobs are unfolded to detail view and queues containing no jobs are just shown as general instance ? ( yes i know that one is a bit greedy , i just asked for tips ;-) ) - last one: when clicking on the Job ID of a running Job the below error messages is displayed. Is there anything i might have configured wrong ? An error has occured java.net.ConnectException: Connection refused Cocoon stacktrace[hide] Failed to process pipeline context://../../../xmlqstat/sitemap.xmap - 119:34 context://../../../xmlqstat/sitemap.xmap - 115:46 context://../../../xmlqstat/sitemap.xmap - 114:66 context://sitemap.xmap - 939:57 all other views work TIA Michael From brendon.oliver at gmail.com Tue Oct 10 17:28:52 2006 From: brendon.oliver at gmail.com (Brendon Oliver) Date: Wed, 11 Oct 2006 07:28:52 +1000 Subject: [xml-qstat-d] xml-qstat error / refresh / customize In-Reply-To: <20061010144430.GA18569@winter.science-computing.de> References: <20061010144430.GA18569@winter.science-computing.de> Message-ID: <200610110728.53091.brendon.oliver@gmail.com> Hi Michael, On Wednesday 11 October 2006 00:44, Michael Arndt wrote: > -how a automatic refesh of the page is done best ? Look for the files qstat-xhtmlCSS-standard-v2.xsl and qstat-xhtmlCSS-terse-v2.xsl (they're in the xsl/ dir). Edit each file, search for the tag, then a few lines below that you will see something like: add another tag directly below that: 120 = refresh every 120 seconds - change the value to whatever suits you. Can't help you with the queue display I'm afraid, except to say that I do like the idea of being able to collapse/expand queue instances to save screen real estate (something I've been wanting to look into doing myself, just no time to do it). > - last one: when clicking on the Job ID of a running Job the below error > messages is displayed. Is there anything i might have configured wrong ? That sounds to me like you haven't got the CGI script setup which runs qstat when you select a specific job. I posted a rather brief explanation of this on the Grid Engine users mailing list in mid-September. Here's a snippet of the email, and some followup info posted by Chris (hope it helps): My email: > Have a look in the xmlqstat/sitemap.xmap file, you will find a section > like: > > > > > > > > > > > [ Edit: I think the reference to xmldata/sgeJob (above) needs to be xmldata/sgeJob.cgi - but I'm sure you can firugre it out ;-) ] > The important part is that URL - it is effectively a callback to the > webserver listening on port 8889 on the loopback address. You need to > configure your webserver so that something is actually listening there. If > your xml-qstat install is on a different machine to the webserver, then > you'll need to change the loopback address accordingly. In my setup, I > just added a virtual host to apache listening on that port and configured > to allow the > xml-qstat/scripts dir to be treated as CGI scripts: > > <--- START HERE ---> > Listen 8889 > > # Virtual host for xml-qstat Job CGI script: > > DocumentRoot /usr/local/xml-qstat/ > ErrorLog /usr/local/xml-qstat/logs/error.log > CustomLog /usr/local/xml-qstat/logs/access.log common > > ScriptAlias /xmldata/ "/usr/local/xml-qstat/scripts/" > > > AllowOverride None > Options ExecCGI > Order allow,deny > Allow from all > > > > # Allow the main server to proxy requests to xmlqstat: > ProxyPass /xmlqstat http://127.0.0.1:8888/xmlqstat > ProxyPassReverse /xmlqstat http://127.0.0.1:8888/xmlqstat > <--- END HERE ---> > > I added the ProxyPass directives because we're using apache to proxy > requests to the cocoon server. Obviously, you'll need to adjust the paths > show above to suit your installation. Chris' followup: > On Friday 15 September 2006 07:10, Chris Dagdigian wrote: > Wow, people are actually using xml-qstat! > > I should mention here that all this ugliness is required because > whenever someone wants detailed job information we need to do a > callback to a program or process that can do a "qstat -xml -j > " command and pipe the resulting XML back into the XSLT engine > that paints the XHTML into the browser. > > Right now I do this by supplying a cheezy perl script that can be > dropped into an apache cgi-bin directory. Then we configure the > Cocoon sitemap.xmap file (as Brendon mentions) to let the application > know that the source for job-level XML data is actually another URL > location. > > The *real* way to do this using the Cocoon XML publishing framework > is to build what is called a "Cocoon XML generator" -- this is a Java > program or class that "knows" how to call "qstat -j -xml " in > the proper way. This generator is then configured into the > application server as another XML data source that is called upon > whenever someone needs qstat data. > > I did not do this because I'm a horrible Java coder. I can handle XML > and the XSLT transformations into XHTML/CSS but I'm unable to write a > Cocoon generator for SGE 'qstat' > > Consider this an open invitation to the list -- is there anyone that > knows java/Cocoon and is interested in creating a generator for qstat > data? > > -Chris -- Govern a great nation as you would cook a small fish. Don't overdo it. -- Lao Tsu 07:09:05 up 6 days, 14:33, 3 users, load average: 0.25, 0.17, 0.21 From dag at sonsorol.org Tue Oct 10 18:08:33 2006 From: dag at sonsorol.org (Chris Dagdigian) Date: Tue, 10 Oct 2006 18:08:33 -0400 Subject: [xml-qstat-d] xml-qstat error / refresh / customize In-Reply-To: <200610110728.53091.brendon.oliver@gmail.com> References: <20061010144430.GA18569@winter.science-computing.de> <200610110728.53091.brendon.oliver@gmail.com> Message-ID: <53EF9F05-3C08-414C-B137-E5671B74775F@sonsorol.org> Thanks Brendon for answering this! Looks like the meta-refresh thing should be first on the list for SVN code checkins. Would people prefer the simple hard-coded refresh value like Brandon mentions or should it be something that the user can set/adjust within the web interface? -Chris On Oct 10, 2006, at 5:28 PM, Brendon Oliver wrote: > Hi Michael, > > On Wednesday 11 October 2006 00:44, Michael Arndt wrote: > >> -how a automatic refesh of the page is done best ? > > Look for the files qstat-xhtmlCSS-standard-v2.xsl and > qstat-xhtmlCSS-terse-v2.xsl (they're in the xsl/ dir). > > Edit each file, search for the tag, then a few lines below > that you > will see something like: > > > > add another tag directly below that: > > > > 120 = refresh every 120 seconds - change the value to whatever > suits you. > > Can't help you with the queue display I'm afraid, except to say > that I do like > the idea of being able to collapse/expand queue instances to save > screen real > estate (something I've been wanting to look into doing myself, just > no time > to do it). > >> - last one: when clicking on the Job ID of a running Job the below >> error >> messages is displayed. Is there anything i might have configured >> wrong ? > > That sounds to me like you haven't got the CGI script setup which > runs qstat > when you select a specific job. I posted a rather brief > explanation of this > on the Grid Engine users mailing list in mid-September. Here's a > snippet of > the email, and some followup info posted by Chris (hope it helps): > > My email: >> Have a look in the xmlqstat/sitemap.xmap file, you will find a >> section >> like: >> >> >> >> >> >> >> >> >> >> >> > [ Edit: I think the reference to xmldata/sgeJob (above) needs to be > xmldata/sgeJob.cgi - but I'm sure you can firugre it out ;-) ] >> The important part is that URL - it is effectively a callback to the >> webserver listening on port 8889 on the loopback address. You >> need to >> configure your webserver so that something is actually listening >> there. If >> your xml-qstat install is on a different machine to the webserver, >> then >> you'll need to change the loopback address accordingly. In my >> setup, I >> just added a virtual host to apache listening on that port and >> configured >> to allow the >> xml-qstat/scripts dir to be treated as CGI scripts: >> >> <--- START HERE ---> >> Listen 8889 >> >> # Virtual host for xml-qstat Job CGI script: >> >> DocumentRoot /usr/local/xml-qstat/ >> ErrorLog /usr/local/xml-qstat/logs/error.log >> CustomLog /usr/local/xml-qstat/logs/access.log common >> >> ScriptAlias /xmldata/ "/usr/local/xml-qstat/scripts/" >> >> >> AllowOverride None >> Options ExecCGI >> Order allow,deny >> Allow from all >> >> >> >> # Allow the main server to proxy requests to xmlqstat: >> ProxyPass /xmlqstat http://127.0.0.1:8888/xmlqstat >> ProxyPassReverse /xmlqstat http://127.0.0.1:8888/xmlqstat >> <--- END HERE ---> >> >> I added the ProxyPass directives because we're using apache to proxy >> requests to the cocoon server. Obviously, you'll need to adjust >> the paths >> show above to suit your installation. > > Chris' followup: >> On Friday 15 September 2006 07:10, Chris Dagdigian wrote: >> Wow, people are actually using xml-qstat! >> >> I should mention here that all this ugliness is required because >> whenever someone wants detailed job information we need to do a >> callback to a program or process that can do a "qstat -xml -j >> " command and pipe the resulting XML back into the XSLT engine >> that paints the XHTML into the browser. >> >> Right now I do this by supplying a cheezy perl script that can be >> dropped into an apache cgi-bin directory. Then we configure the >> Cocoon sitemap.xmap file (as Brendon mentions) to let the application >> know that the source for job-level XML data is actually another URL >> location. >> >> The *real* way to do this using the Cocoon XML publishing framework >> is to build what is called a "Cocoon XML generator" -- this is a Java >> program or class that "knows" how to call "qstat -j -xml " in >> the proper way. This generator is then configured into the >> application server as another XML data source that is called upon >> whenever someone needs qstat data. >> >> I did not do this because I'm a horrible Java coder. I can handle XML >> and the XSLT transformations into XHTML/CSS but I'm unable to write a >> Cocoon generator for SGE 'qstat' >> >> Consider this an open invitation to the list -- is there anyone that >> knows java/Cocoon and is interested in creating a generator for qstat >> data? >> >> -Chris > > > > -- > Govern a great nation as you would cook a small fish. Don't overdo > it. > -- Lao Tsu > > 07:09:05 up 6 days, 14:33, 3 users, load average: 0.25, 0.17, 0.21 > > _______________________________________________ > xml-qstat-d mailing list > xml-qstat-d at mail.bioteam.net > http://mail.bioteam.net/mailman/listinfo/xml-qstat-d From M.Arndt at science-computing.de Wed Oct 11 01:55:48 2006 From: M.Arndt at science-computing.de (Michael Arndt) Date: Wed, 11 Oct 2006 07:55:48 +0200 Subject: [xml-qstat-d] xml-qstat error / refresh / customize In-Reply-To: <53EF9F05-3C08-414C-B137-E5671B74775F@sonsorol.org> References: <20061010144430.GA18569@winter.science-computing.de> <200610110728.53091.brendon.oliver@gmail.com> <53EF9F05-3C08-414C-B137-E5671B74775F@sonsorol.org> Message-ID: <20061011055548.GA31321@winter.science-computing.de> Hello Brendon,Chris first thanks to Brendon for friendly and perfectly answering my? questions, i think that should more than suffice to solve both issues :-). Voting Refresh global or user specific: -a static, global refresh like presented for me is completely sufficient. "Typical" job throughput most times depends on the kind of calcuations done ( monte carlo / banking, cae simulation / engineering crunching molecules / your bussines ) so i assume refresh most times is a site specific compromiss. thx Micha From M.Arndt at science-computing.de Sun Oct 22 14:39:17 2006 From: M.Arndt at science-computing.de (Michael Arndt) Date: Sun, 22 Oct 2006 20:39:17 +0200 Subject: [xml-qstat-d] xmlqstat / coccon / /tmp/Jetty* In-Reply-To: <20061010144430.GA18569@winter.science-computing.de> References: <20061010144430.GA18569@winter.science-computing.de> Message-ID: <20061022183917.GA2116@winter.science-computing.de> Hello * Either xmlqstat via cocoon or xml-qstat-cacher or Cocoon is generating Directories: /tmp/Jetty* containing very large content wrt to a /tmp Dir: 2-3 GB Size leading to a full rootdisk Questions: -can i configure the tools to use another directory for caching ? -is it Coccon or xmlqstat that generates those Jetty Directories ? TIA Micha