Register forum user name Search FAQ

Gammon Forum

Notice: Any messages purporting to come from this site telling you that your password has expired, or that you need to verify your details, confirm your email, resolve issues, making threats, or asking for money, are spam. We do not email users with any such messages. If you have lost your password you can obtain a new one by using the password reset link.
 Entire forum ➜ MUSHclient ➜ Suggestions ➜ Automatic log file size limiting and splitting?
Automatic log file size limiting and splitting?

It is now over 60 days since the last post. This thread is closed.     Refresh page

Posted by Weston   (28 posts)  Bio
Date Fri 14 May 2004 12:04 AM (UTC)
Message
I usually have logging enabled and it's great, but at the end of the day I notice that each of my log files are HUGE.
Opening a TXT or (even more so) a HTML file that is over 3 megabytes is rather taxing on my computer and (most likely) the computers of other MU* players.

My suggestion:
Would it be possible to sit up a filesize limit?
It would work like this: when the log file being written hits the size limit (i.e. 500kb) that file would be ended and the logging would be written to a new file, a second file. Then If that second file also grew to (once again, for example) 500kb, it would also be ended and a 3rd file would be started.

What do you think?
Top
Posted by Flannel   USA  (1,230 posts)  Bio
Date Reply #1 on Fri 14 May 2004 01:15 AM (UTC)
Message
You could easily script something to check the file size (FSO: obj.size) and then append the closing HTML tags, and open a new one. You could even make it add a link to the next page at the end.
~Flannel

Messiah of Rose
Eternity's Trials.

Clones are people two.
Top
Posted by David Haley   USA  (3,881 posts)  Bio
Date Reply #2 on Fri 14 May 2004 09:28 AM (UTC)

Amended on Fri 14 May 2004 09:29 AM (UTC) by David Haley

Message
If you're running on an NTFS filesystem (the default in WinXP and Win2000 I believe), then you can use the option 'compress folder to save disk space' on your log folder. To do so, right click on the folder in explorer, click 'advanced', and the option is there. Apply to folder, subfolders and all files.

I have 106mb of log files uncompressed - I'm a coder, not a player, so I don't have as many logs in game. :) Using this option, the logs take up only 32.6 mb. Depending on your log files (you might have much more repetitive text such as room descriptions or exit lists), the compression ratio may be even better.

Note that compressed files like this are not "zipped" and can be accessed just like any old file on your hard drive. Most importantly, MUSHclient can write to them 'live' and you don't have to do anything at all. The (de)compression is fast and completely automatic.

(edit)
Eh-heh. Sorry, I didn't notice that you were talking about opening the files, not storing them... yes, that is definitely a problem even for faster computers. Opening up a multi-megabyte HTML file is always rather tough on the computer. :)
David Haley aka Ksilyan
Head Programmer,
Legends of the Darkstone

http://david.the-haleys.org
Top
Posted by Nick Gammon   Australia  (23,121 posts)  Bio   Forum Administrator
Date Reply #3 on Sat 15 May 2004 02:29 AM (UTC)
Message
A simple approach would be to make a timer that fires every hour or so and closes and re-opens the log file. If you open it with a blank name it should take the name from the configuration window, which can include the date/time.
- Nick Gammon

www.gammon.com.au, www.mushclient.com
Top
Posted by Poromenos   Greece  (1,037 posts)  Bio
Date Reply #4 on Sat 15 May 2004 11:07 AM (UTC)
Message
Also, not to advertise or anything, but I've found that uhh, what's its name... TextPad can open my multi-megabyte logs (200+) without any difficulty at all... I use it for browsing through the logs, maybe you could give it a shot.
Vidi, Vici, Veni.
http://porocrom.poromenos.org/ Read it!
Top
Posted by Flannel   USA  (1,230 posts)  Bio
Date Reply #5 on Sun 16 May 2004 07:23 AM (UTC)
Message
Textpad doesnt parse HTML. Or rather, it doesnt display/interperate/whatever HTML.

HTML logs are really the only problematic thing, plaintext is no big deal for large files.
~Flannel

Messiah of Rose
Eternity's Trials.

Clones are people two.
Top
Posted by Meerclar   USA  (733 posts)  Bio
Date Reply #6 on Sun 16 May 2004 08:02 AM (UTC)

Amended on Sun 16 May 2004 08:09 AM (UTC) by Meerclar

Message
I use TextPad for code writing when I need to deal with java or html or ANYTHING script related because it can (and does) match tags and whatnot quite nicely. Mind you, we aren't talking about the windows textpad which is um..... crap but a much more powerful downloaded editor with pretty nearly the same name. You may need to change settings a bit to get it to display code tags properly but it will do so, can even compile java for that matter.

The TextPad I'm refering to is put out by helios software and can be found at www.textpad.com and should afaik be available in an unlimited fully functional eval version.
Meerclar - Lord of Cats
Coder, Builder, and Tormenter of Mortals
Stormbringer: Rebirth
storm-bringer.org:4500
www.storm-bringer.org
Top
Posted by David Haley   USA  (3,881 posts)  Bio
Date Reply #7 on Sun 16 May 2004 08:21 AM (UTC)
Message
gVim! *Real* coders use gVim! :) (Or maybe xEmacs...)

http://www.vim.org/download.php

There's a Windows version available that works very, very nicely.

If gVim is scary to you (understandable if you're not used to *nix environments), Crimson Editor is also a pretty good text editor.

http://www.crimsoneditor.com/

Don't know TextPad, so can't say much about it.
David Haley aka Ksilyan
Head Programmer,
Legends of the Darkstone

http://david.the-haleys.org
Top
Posted by Flannel   USA  (1,230 posts)  Bio
Date Reply #8 on Sun 16 May 2004 08:37 AM (UTC)
Message
Eh, for that sort of stuff (the matching and such) you can use Arachnophilia. Completely customizable, although it is in java.

For macros and other massive replacements and such, I use PFE. I still havent found anything with a macro interface that I like besides it.


But point being guys, were not talking about text editors for text files, or even html files, were talking about VIEWING HTML, as in, <b> makes it bold, and such. Since, when you log in HTML, most of the time you dont view it in the HTML form, but as a webpage.

And he's looking for something distributable, shouldnt need to download anything. He just wants to be able to post logs on websites, or whereever, without spazzing out the unsuspecting users computer with a massive HTML file.
~Flannel

Messiah of Rose
Eternity's Trials.

Clones are people two.
Top
Posted by David Haley   USA  (3,881 posts)  Bio
Date Reply #9 on Sun 16 May 2004 08:57 AM (UTC)
Message
If you have a massive HTML file, well, you're pretty much screwed. Any program will just take a long time to render it (that's the term for what you're talking about). That's just how life goes... it's completely dependent on your computer speed.

The only option if you want to put it on websites, and if you don't want to have a massive HTML file, is to split it. There may be programs that do that.

I googled for 'html splitter' and the very first result looks promising:

http://www.rekenwonder.com/htmlsplitter.htm
David Haley aka Ksilyan
Head Programmer,
Legends of the Darkstone

http://david.the-haleys.org
Top
Posted by Poromenos   Greece  (1,037 posts)  Bio
Date Reply #10 on Sun 16 May 2004 12:31 PM (UTC)
Message
Ah, I hadn't realized that we were talking about viewing html logs... I guess i didn't read that through :p I agree with Ksilyan on this one, it will take a lot of time, but you could use your text editor to split the file and add the header/footer to the new files and view those... By the way, Flannel, how do you replace stuff in PFE? I don't know if I have enabled it, but send me an email through the forum :p I find PFE's lack of regex etc makes it hard to do mass replacements, but I haven't looked into macros... Can you do more complex replacements than the "replace" dialog?
Vidi, Vici, Veni.
http://porocrom.poromenos.org/ Read it!
Top
Posted by Nick Gammon   Australia  (23,121 posts)  Bio   Forum Administrator
Date Reply #11 on Sun 16 May 2004 09:21 PM (UTC)
Message
Yes, I would not post multi-megabyte log files for online viewing. But that is why the question was asked in the first place. :)

Probably a simple way would be to make a timer that fires every hour or two simply close and re-open the log, with the date/time in the name.

That way you always get (say) 24 logs per day, which makes more sense anyway when you are organising your web pages.

If they end up being too small, make it every two hours.

Alternatively have the timer do what Flannel suggested and have it find the log file size and close and re-open it based on that.
- Nick Gammon

www.gammon.com.au, www.mushclient.com
Top
Posted by Flannel   USA  (1,230 posts)  Bio
Date Reply #12 on Mon 17 May 2004 01:36 AM (UTC)
Message
Or you could go through afterwards (with FSO and a small script, with some intelligence) and cut off every XXX lines, insert closing tags (and a link to the next file), and make a new file with appropriate opening tags with the rest.

You could do it either way, make a new file for the stuff after the split, or beforehand (it would be less to eat away and the stuff you eat, you put into a new file, rather than the other way around, less movement).
~Flannel

Messiah of Rose
Eternity's Trials.

Clones are people two.
Top
Posted by Weston   (28 posts)  Bio
Date Reply #13 on Wed 19 May 2004 10:55 PM (UTC)
Message
Thanks for all of the replies! (Even the ones that don't apply to my problem :).

The solution I like most is that one where a timer would check the size every hour or so and then close and re-open the log, with the date/time in the name.

I now have another question:


Where and how can I ask for some help with that... I have no idea how to write a script.
Top

The dates and times for posts above are shown in Universal Co-ordinated Time (UTC).

To show them in your local time you can join the forum, and then set the 'time correction' field in your profile to the number of hours difference between your location and UTC time.

33,470 views.

It is now over 60 days since the last post. This thread is closed.     Refresh page

Go to topic:           Search the forum

[Go to top] top

Information and images on this site are licensed under the Creative Commons Attribution 3.0 Australia License unless stated otherwise.

Home