Uniface getting slow-motion with bigger Strings

Author: hansen@ahp-gmbh.de (-GHAN-)

Hi everybody, i'm using the uxmlwriter to create XML-streams to transport occurences from my Entity from one component to another. Works well so far, but as the number of records grow, uniface goes slow-motion! Retrieving a 1000 Records of an Entity containing 180 fields takes (on this machine) 1.5 minutes! The volume of the data is approx. 5.5MB. The Code does nearly the same as the xmlwriter does, so you can test it yourselves by doing: -[CODE follows]----- putmess "start : %%$clock" $1=1 while($1 < 1000) $2 = 1 while ($2 < 180) vString = "%%vString%%%XMLTEEEEEEEEEEXT"; ### simulating a XML-Tag $2 = $2 + 1 endwhile $1 = $1 + 1 endwhile putmess "End : %%$clock" -[CODE ends here]--- You now might say "Dude, get your data in shape! Nobody needs all these records ever!" and that might be true. (in fact i just got a call containing a similiar message. Thx 2 A.H.) But when you need to transfer data using xml then Uniface internally uses a STRING-CLASS (of a sort) to hold the data! Afterwards you transfer the data in blocks to where you want it ... lets say a STRING !? ;) While using XMLSave i didnt experience this performance slowdown! I tried to trick Uniface by declaring my target variable to "xmlstream" but this didn't help it ;) And now don't tell me to use XMLSave instead ;) ... this wont solve my problem !!! So ... does anybody know about this? best Regards -GHAN-

5 Comments

  1. Hi, this is a very well known "problem" with UNIFACE strings, which is deeply embedded in the way strings are handled in the Kernel. The "trick" is not to let the string grow too much: just use a filedump/append now and then to offload a big chunk of the string and start with a "clean" string again. At the end, just file_load the longstring. Note: filedump/append has some overhead as well, so it pays off only when the chunks are big enough. Its worth a test. Success, Uli


    Author: ulrich-merkel (ulrichmerkel@web.de)
  2. Moin Ulrich, its allways nice to get feedback on posted stuff. Strings are definatly the bottlenek ... ! Using a file is of course a workaround, but that won't do when using xml for reading and transferring entity-data (lager than 10kb in a stateless environment). This puts me back to windows 3.1 where 80 percent of your harddrive turned to be .ini and .tmp files ;))) *bwaaaaaah* We gave the filedump-thing a try lately and tested the same functionality with and selfmade-3gl which handled the data. Outstanding performance ... but as i already mentioned this would demand a huge amount of files, wellorganised and after all we move away from the "normal" way to handle data :) Just imagine how many temp-files you produce while working with 5 entities (read and write) multiplied by lets say 25 clients at same time! And having finished that thought: What about the integrity of the data!? Need sensible data? just copy the tempfiles- the wellformed xml will make it easy! :))) ... Maybe i should build another 3gl which only samples large-strings and returns them in ONE piece ... Thx for reading this -GHAN-


    Author: -GHAN- (hansen@ahp-gmbh.de)
  3. Hi G?nter Here is a modification of your example with a nearly linear performance Trigger from Form: A_LONGTEXT ****** Y4:: ****** entry LONG1 1 variables 2 string vString 3 endvariables 4 X_TEXT = "%%X_TEXT%%%Long1 start : %%$clock" 5 $1=1 6 while($1 < 1000) 7-1 $2 = 1 8-1 while ($2 < 180) 9-2 vString = "%%vString%%%XMLTEEEEEEEEEEXT"; ### simulating a XML-Tag 10-2 $2 = $2 + 1 11-2 endwhile 12-1 $1 = $1 + 1 13-1 message/hint "%%$clock%%% %%$1%%%" 14-1 endwhile 15 X_TEXT = "%%X_TEXT%%% End : %%$clock%%^" 16 end ; long1 ****** entry LONG2 1 variables 2 string vString, vBuffer 3 endvariables 4 X_TEXT = "%%X_TEXT%%%Long2 start : %%$clock" 5 $1=1 6 while($1 < 1000) 7-1 $2 = 1 8-1 vString = "" 9-1 while ($2 < 180) 10-2 vString = "%%vString%%%XMLTEEEEEEEEEEXT"; ### simulating a XML-Tag 11-2 $2 = $2 + 1 12-2 endwhile 13-1 vBuffer = "%%vBuffer%%%%%vString%%%" 14-1 $1 = $1 + 1 15-1 message/hint "%%$clock%%% %%$1%%%" 16-1 endwhile 17 X_TEXT = "%%X_TEXT%%% End : %%$clock%%^" 18 end ; long2 if I have some time, i will check if the same improvement can be achieved working with the UXMLWRITER. Success, Uli


    Author: ulrich-merkel (ulrichmerkel@web.de)
  4. Hi Chang, with UXMLWRITER the thing is that it allocates more memory on the fly once the xml document become bigger and bigger. The first time you generate such a big XML document (5.5Mb) it really needs a lot of time to allocate this memory space. A second time you use the same instance of the UXMLWRITER component it should be a lot faster. So, if you can, always reuse the UXMLWRITER instance for your (big) XML write actions. Maybe this helps.


    Author: Gerton Leijdekker (gerton.leijdekker@uniface.com)
  5. Thanx for the hint, Gerton! But the reading Component is a standalone Service, delivering the XMLdata to anyone, who requests for it. Stateless Environment! And so ... this won't do :) I guess, there is no real solution for this Problem. But while talking about allocation ... how does XMLLOAD/SAVE handle this? those two never seem to slow down! regards to the netherlands -GHAN-


    Author: -GHAN- (hansen@ahp-gmbh.de)