Samson, in case you are reading this, can I suggest a fix to FUSS?
I am playing with writing a few snippets, and noticed this in fight.c:
void new_dam_message( CHAR_DATA * ch, CHAR_DATA * victim, int dam, unsigned int dt, OBJ_DATA * obj )
{
char buf1[256], buf2[256], buf3[256];
... (and further on) ...
snprintf( buf1, 256, "$n %s $N%c", vp, punct );
snprintf( buf2, 256, "You %s $N%c", vs, punct );
snprintf( buf3, 256, "$n %s you%c", vp, punct );
Wouldn't it be much safer to do this:
snprintf( buf1, sizeof buf1, "$n %s $N%c", vp, punct );
snprintf( buf2, sizeof buf2, "You %s $N%c", vs, punct );
snprintf( buf3, sizeof buf3, "$n %s you%c", vp, punct );
This is safer, in case you ever change the sizes of buf1 etc., but removes the "magic number" of 256, and replaces it with a proper explanation for what that paremeter really is.
There are similar instances further down the function.
So is that supposed to be "sizeof buf1" and not "sizeof( buf1 )"?
And would this only be necessary for places like the above, or would it be more or less everywhere? I don't mind having this done but it wouldn't happen quickly with the limited time I have available these days - unless someone were to provide a patch against the latest version :)
I don't think you need the parens, even though most people put them in.
And you'd use it every place you used the hard-coded, numerical size of a buffer. The idea is to pull the size directly from the compiler's information, which can't really be wrong, instead of depending on the programmer to update all places the size is referenced if the buffer size is changed.
This is crying out for an automated fixup with regular expressions, which I am almost ready to release. However this caught my eye while I was doing it:
Samson, I'm assuming you have access to (or use) MUSHclient? I wrote this script using the Immediate window in MUSHclient, which will fixup the entire FUSS source directory - writing to a new directory for safety. You can do a diff on the two directories to check the changes look reasonable, before copying the changed files back over the other ones.
This is the script:
-- file paths, amend as required
olddir = "C:\\smaugfuss\\src\\"
newdir = "C:\\smaugfuss\\src_fixed\\"
-->> see below for improvements to this bit
standard_replace = "snprintf( %1, sizeof (%1),"
replacements = {
{ find = "snprintf%s*%(%s*([A-Za-z0-9_]+)%s*,%s*[0-9]+%s*,",
repl = standard_replace
},
{ find = "snprintf%s*%(%s*([A-Za-z0-9_]+)%s*,%s*MAX_[A-Z]+_LENGTH%s*,",
repl = standard_replace
},
{ find = "snprintf%s*%(%s*([A-Za-z0-9_]+)%s*,%s*SMST%s*,",
repl = standard_replace
},
{ find = "snprintf%s*%(%s*([A-Za-z0-9_]+)%s*,%s*LGST%s*,",
repl = standard_replace
},
{ find = "snprintf%s*%(%s*([A-Za-z0-9_]+)%s*,%s*IMC_BUFF_SIZE%s*,",
repl = standard_replace
},
} -- end replacements table
-->> end of part that has been improved (copy from below)
function process_file (name)
local f, s
local total = 0
print ("Processing: " .. name)
f = assert (io.open (olddir .. name, "rb")) -- open input
s = f:read ("*a") -- read all of it
f:close () -- close it
-- apply a series of changes
for k, v in ipairs (replacements) do
local count
s, count = string.gsub (s, v.find, v.repl)
total = total + count
end -- for each replacement
if total > 0 then
print (" -- " .. total .. " changes made.")
f = assert (io.open (newdir .. name, "wb")) -- open output
f:write (s) -- write to it
f:close () -- close that file now
end -- changes made
return total
end -- process_file
-- START HERE --
local grand_total = 0
local t = assert (utils.readdir (olddir .. "*.c"))
local k
for k in pairs (t) do
grand_total = grand_total + process_file (k)
end -- for
print ("** Total of " .. grand_total .. " changes made.")
After running this (you will need to change the directory locations to match your setup) I got this:
I can just send you the diff if you prefer, but this gives you the tool to make similar changes yourself.
What the script does is use MUSHclient's Lua extension utils.readdir to get a list of all the .c files in the directory, and then opens each one and applies a series of string.gsub to it. The main one is to look for the case:
snprintf (buf, (some number)
... and replace "(some number)" by sizeof "(buf)".
However there are a few special cases that seem to occur frequently, like MAX_STRING_LENGTH, SMST, and LGST, so I have added those as extras.
There are still a handful of cases this script doesn't handle (52 of them), along the lines of:
With a slightly more complex regexp you can handle the cases of the buf + strlen (buf). This part replaces the section above which defines the regular expressions:
The extra parts (that use standard_replace2) now check for "+ strlen (%1)" which is the case of taking the strlen of whatever we are doing the snprintf to, and using that for a slightly different sort of replacement.
After running that on the entire source, there are only 19 special cases that need manually fixing, for example:
So all I'd need to do is have this script run inside of mushclient and it will patch up the smaug source to change all of the buffer usage in the code? And how will I be able to spot the remaining special cases?
Also going to need to know what the "Immediate window" is in Mushclient since I can't find a menu option called anything like that.
This program fails to compile, with a parse error at the first printf. But it works with parens around the int for the sizeof; so it appears that parens are indeed only necessary for a type. I didn't test for structures etc., though.
So is that supposed to be "sizeof buf1" and not "sizeof( buf1 )"?
My initial suggestion omitted the brackets, however because of the confusion I put them into the script. Whichever you prefer. I don't think they do any harm.
Quote:
So all I'd need to do is have this script run inside of mushclient and it will patch up the smaug source to change all of the buffer usage in the code?
Yes, although since MUSHclient is a Windows program, unless you are using Cygwin, you would need to copy the source to a suitable directory, put its path into the script, and run it. Make a blank directory for the src_fixed (output files) so you can run a diff on the before and after.
Quote:
And how will I be able to spot the remaining special cases?
As I mentioned, this command in Linux or Cygwin will do it:
grep -n snprintf *.c | grep -v sizeof
That will give you the files and line numbers, namely:
Error number: 0
Event: Run-time error
Description: [string "Immediate"]:51: attempt to index global `io' (a nil value)
stack traceback:
[string "Immediate"]:51: in function `process_file'
[string "Immediate"]:80: in main chunk
Called by: Immediate execution
The only thing relating to scripting I see in the world properties that seems to be relevant is the "enable" checkbox. Which I have enabled. I don't see anything about sandboxing or enabling system libraries.
It might be in global preferences, then. But I know there's an input field for sandboxing somewhere. Sorry to not be more specific, I'm at work at don't have the client in front of me.
The "io" library is sandboxed out (suppressed) by default for the Lua scripting engine, as doing thing like io.open of an output file could be used to inject viruses or trojan horses into a target system. The intention is that you allow the io library for trusted plugins, or your own code.
Fist off, thanks for making this available. As Samson reported, it works beautifully!
I suppose a natural question at this point would be about the mudstrlcpy and mudstrlcat calls, which are only slightly different in how they are set up, but could be made safer in the same fashion.
Is it difficult to expand this script to encompass those cases as well? I was tempted to go in and correct them by hand to use sizeof() but when I grepped to jog my memory of just how many we were talking about, I figured it might be worthwhile to ask about changing the script first. ;)
Again, thanks for putting this together and making it available to the community! :-)
That would assume this can be done on strcpy/strcat and related type functions. I got the feeling it's only really useful on sprintf type functions. But hell, what do I know about strings, I could be wrong :)
Noticing now that something may have been improperly replaced with this. I found that the prompt was not displaying properly and there were other issues in unrelated areas as well. Has anyone else had similar issues? I'm seeing if I can track down the problem, but so far I've not figured out exactly what is causing it. I figure it might be one bad replacement somewhere, but I'm not sure yet.
I haven't noticed any problems with standard SmaugFUSS. The prompt looks OK. Can you be more specific about "other issues?". Did you just do the initial changes I recommended or make some manual ones as well, or maybe the mudstrlcpy calls you talked about?
I can however see a serious problem. Take this test program:
#include <stdio.h>
int main ()
{
char buf [1000];
char * buf2 = (char *) malloc (1000);
printf ("buf is %i bytes.\n", sizeof (buf));
printf ("buf2 is %i bytes.\n", sizeof (buf2));
return 0;
}
Both buf and buf2 represent 1000 bytes of memory. However look at the output:
buf is 1000 bytes.
buf2 is 4 bytes.
It has correctly reported that buf2 (the pointer) is 4 bytes long, however from the point of view of using it in a printf or similar it is really (pointing to) 1000 bytes.
I can't see a simple solution to this, but to manually review each of the changes to make sure that the change is for a locally-declared buffer and not a pointer passed down to the function.
I found Splint at http://www.splint.org/ which seems to do a reasonable job in general, however it does not detect this particular problem.
I can't offhand think of a way of automating this. However I think it will "fail safe" - that is, instead of taking the correct buffer size will take 4 (the size of a pointer) so it won't probably lead to a crash, and if you happen to see a display where there are only 4 characters, you have probably stumbled across a case in point.
So if this hasn't been done yet would it be advisable to hold off and wait? The script makes a great many changes and I'd really rather not have to manually verify them all :/
Much as I hate to say it, I think that would be wise. It is hard to be certain the changes will always be for the better, bearing in mind the possible problem with pointers rather than fixed buffers. I notice while researching this, this rather strange sequence, in act_comm.c starting at line 469:
This code (before my changes) just seems plain wrong. The snprintf is to a buffer of 4096 bytes (MAX_STRING_LENGTH) but the actual memory available is 1028 bytes (MAX_INPUT_LENGTH + 4).
I wanted to get this project finished, so I wrote another bit of Lua code to try to detect the cases where the "before" and "after" of my automated changes were not identical, and then follow those ones up.
This slightly hacky bit of code below does the trick:
-- pattern to detect a function declaration
pat = "[%a_]+%s+%*?%s*([%a%d_]+)%s*%b().-(%b{})"
olddir = "C:\\smaugfuss\\src\\"
snprintf1 = "snprintf%s*%(%s*([%a%d_]+)%s*,%s*("
snprintf2 = "snprintf%s*%(%s*([%a%d_]+)%s*%+%s*strlen%s*%(%s*%1%s*%)%s*,%s*("
found = 0
not_found = 0
replacements = {
{ find = snprintf1 .. "%d+)%s*,",
},
{ find = snprintf1 .. "MAX_%u+_LENGTH)%s*,",
},
{ find = snprintf1 .. "SMST)%s*,",
},
{ find = snprintf1 .. "LGST)%s*,",
},
{ find = snprintf1 .. "IMC_BUFF_SIZE)%s*,",
},
{ find = snprintf2 .. "IMC_BUFF_SIZE)%s*%-%s*strlen%s*%(%s*%1%s*%)%s*,",
},
{ find = snprintf2 .. "%(?%s*MAX_%u+_LENGTH)%s*%-%s*strlen%s*%(%s*%1%s*%)%s*%)?%s*,",
},
{ find = snprintf2 .. "LGST)%s*%-%s*strlen%s*%(%s*%1%s*%)%s*,",
},
} -- end replacements table
function check_arg (s, size)
match = string.find (saved_body,
"char%s+" .. s .. "%s*%[%s*" ..
size .. "%s*%]")
if match then
found = found + 1
else
not_found = not_found + 1
key = s .. ":" .. size
if not reported [key] then
reported [key] = true
ColourNote ("white", "red",
"Cannot find declaration for " .. s .. " (size " .. size ..
") in function " .. saved_name
)
end -- if not already reported it
end -- if
end -- check_arg
function process_function (name, body)
-- Note (" +++ function: " .. name)
reported = {}
saved_name = name -- in case we need to report it
saved_body = body -- move to global scope for check_arg
-- check each change
for k, v in ipairs (replacements) do
s, count = string.gsub (body, v.find, check_arg)
file_count = file_count + count
end -- for each replacement
end -- process_function
function process_file (name)
local f, s
file_count = 0
print ("Processing: " .. name)
f = assert (io.open (olddir .. name, "rb")) -- open input
s = f:read ("*a") -- read all of it
f:close () -- close it
-- fix up imc strange function defs
s = string.gsub (s, "PFUN%( ([%a%d_]+) %)",
"void %1( IMC_PACKET *q, char *packet )")
s = string.gsub (s, "IMC_CMD%( ([%a%d_]+) %)",
"void %1( CHAR_DATA *ch, char *argument )")
s = string.gsub (s, "'}'", "") -- confuses function finding
string.gsub (s, pat, process_function)
print (" -- " .. file_count .. " changes found")
end -- process_file
print (string.rep ("-", 60))
local t = assert (utils.readdir (olddir .. "*.c"))
local k
for k in pairs (t) do
process_file (k)
end -- for
Note ("Matches found = ", found)
Note ("No declaration found = ", not_found)
What this is trying to do is break up each file into a function declaration and body of the general form:
return_type name ( args ) { body }
It then scans each function body for the "snprintf" declarations, and for each individual one, then rescans the function body to try to find a local declaration of the form:
Then the name is "buf" and the size is "MAX_STRING_LENGTH".
So, it looks for:
char buf [MAX_STRING_LENGTH]
If it finds this (bearing in mind it must be inside the same function), then it assumes that it was a local declaration, and all is well.
There are a couple of kludgy aspects to this, in particular in the file imc.c a lot of function declarations are done using defines, like this:
PFUN( imc_recv_tell )
Thus there is a bit of pre-processing to put those back into standard function form.
Running this script on my copy of SmaugFUSS gave these results:
Processing: save.c
-- 11 changes found
Processing: act_obj.c
-- 13 changes found
Processing: hashstr.c
-- 3 changes found
Processing: shops.c
-- 24 changes found
Processing: comm.c
Cannot find declaration for pbuf (size MAX_STRING_LENGTH) in function display_prompt
-- 21 changes found
Processing: mud_prog.c
Cannot find declaration for rval (size MAX_STRING_LENGTH) in function isoperator
-- 6 changes found
Processing: planes.c
-- 0 changes found
Processing: skills.c
-- 29 changes found
Processing: fight.c
Cannot find declaration for buf2 (size 256) in function new_dam_message
Cannot find declaration for buf3 (size 256) in function new_dam_message
-- 22 changes found
Processing: act_wiz.c
Cannot find declaration for buf (size ( MAX_STRING_LENGTH) in function do_owhere
Cannot find declaration for buf (size MAX_STRING_LENGTH) in function do_bestow
Cannot find declaration for reboot_time (size 50) in function get_reboot_string
Cannot find declaration for arg2 (size MAX_INPUT_LENGTH) in function do_sedit
Cannot find declaration for buf (size ( MAX_STRING_LENGTH) in function do_ipcompare
-- 88 changes found
Processing: handler.c
-- 7 changes found
Processing: const.c
-- 0 changes found
Processing: mud_comm.c
-- 16 changes found
Processing: act_comm.c
Cannot find declaration for lbuf (size MAX_STRING_LENGTH) in function talk_channel
-- 31 changes found
Processing: imm_host.c
-- 0 changes found
Processing: services.c
-- 2 changes found
Processing: polymorph.c
Cannot find declaration for buf (size MAX_STRING_LENGTH) in function do_morphset
-- 5 changes found
Processing: sha256.c
-- 0 changes found
Processing: boards.c
-- 22 changes found
Processing: act_info.c
Cannot find declaration for buf (size ( MAX_STRING_LENGTH) in function do_exits
-- 34 changes found
Processing: interp.c
Cannot find declaration for buf (size MAX_STRING_LENGTH) in function write_watch_files
Cannot find declaration for cmd_flag_buf (size MAX_STRING_LENGTH) in function check_cmd_flags
-- 10 changes found
Processing: mapout.c
Cannot find declaration for buf (size MAX_STRING_LENGTH) in function count_lines
-- 1 changes found
Processing: imc.c
Cannot find declaration for buf2 (size LGST) in function imclog
Cannot find declaration for buf2 (size LGST) in function imcbug
Cannot find declaration for buf (size LGST) in function imc_recv_tell
Cannot find declaration for buf (size LGST) in function update_imchistory
Cannot find declaration for name (size SMST) in function imc_display_channel
Cannot find declaration for buf (size IMC_BUFF_SIZE) in function imc_recv_who
Cannot find declaration for lbuf2 (size LGST) in function imcfread_config_file
Cannot find declaration for to (size SMST) in function imccommand
Cannot find declaration for buf (size LGST) in function imcsetup
Cannot find declaration for buf1 (size LGST) in function imctell
Cannot find declaration for to (size SMST) in function imcremoteadmin
Cannot find declaration for buf (size LGST) in function imc_send_social
-- 107 changes found
Processing: build.c
Cannot find declaration for buf (size MAX_STRING_LENGTH) in function edit_buffer
-- 57 changes found
Processing: mpxset.c
-- 26 changes found
Processing: magic.c
-- 16 changes found
Processing: hotboot.c
Cannot find declaration for buf2 (size 100) in function do_hotboot
Cannot find declaration for buf3 (size 100) in function do_hotboot
-- 10 changes found
Processing: deity.c
-- 12 changes found
Processing: update.c
-- 16 changes found
Processing: act_move.c
-- 10 changes found
Processing: special.c
-- 7 changes found
Processing: color.c
-- 57 changes found
Processing: misc.c
-- 3 changes found
Processing: makeobjs.c
-- 7 changes found
Processing: reset.c
Cannot find declaration for objname (size MAX_STRING_LENGTH) in function sprint_reset
-- 14 changes found
Processing: comments.c
-- 0 changes found
Processing: mccp.c
-- 0 changes found
Processing: player.c
Cannot find declaration for buf (size MAX_STRING_LENGTH) in function do_statreport
-- 163 changes found
Processing: db.c
Cannot find declaration for buf (size ( MAX_STRING_LENGTH) in function bug
Cannot find declaration for buf (size ( MAX_STRING_LENGTH) in function boot_log
-- 27 changes found
Processing: track.c
-- 8 changes found
Processing: clans.c
-- 13 changes found
Processing: tables.c
-- 7 changes found
Processing: ban.c
-- 4 changes found
Matches found = 834
No declaration found = 75
As you can see there were a handful reported (75 counted but duplicates were suppressed from the display).
I then went through to see if any of the automated conversions were likely to be wrong (ie. converting from the original values to "sizeof (buf)". Most could be accounted for by the fact that the function used this syntax:
In this case "arg2" does not match my test (it is not directly preceded by "char") however it is obviously still correct.
However I have detected 5 cases where I think the actual SMAUG code (in its current form) is wrong. See next post for those.
Nick GammonAustraliaForum Administrator#30
The following are the problem areas I have detected:
comm.c function display_prompt
There is an snprintf like this in that function:
snprintf( pbuf, MAX_STRING_LENGTH, ...
However pbuf is declared like this:
char buf[MAX_STRING_LENGTH];
char *pbuf = buf;
The pointer pbuf is gradually incremented through buf, so the code should probably read:
snprintf( pbuf, sizeof (buf) - strlen (buf), ...
mud_prog.c function mprog_do_ifcheck
A similar thing is happening here with rval, it initially is set to:
char *rval = "";
Later it appears to be incrementing through buf, so I think simply using sizeof (rval) will be definitely wrong, and also MAX_STRING_LENGTH isn't too great either. A bit of reworking is required here.
act_comm.c function talk_channel
As mentioned in an earlier post, lbuf is MAX_INPUT_LENGTH + 4 bytes long, but the snprintf is for MAX_STRING_LENGTH, which is much longer.
The last 2 examples in particular show why it is a good idea to get in the habit of using sizeof, and avoiding typos, rather than copying and pasting numeric lengths (or even defined lengths).
Another way of looking at this is, if you fix the 5 places mentioned above, then wholesale change isn't really necessary, the other places are correct anyway.
However it is "more technically correct" (to change everything) in that you don't have to look at the 900 other instances later on and wonder if the buffer is really the correct size.