Weird weird problem: things overlaping eachother in memory? [SOLVED]

[Baro]

I already asked about this once, and though I had solved it (because I actually had an error in my code that would cause something like that), but it turns out the problem is still there.

Problem is, I believe parts of my code are overwriting parts of the memory they shouldn't, or something similar. Sometimes, when I add more code, I get random errors... In parts of the code that are BEFORE the part I've changed! Right now, if I comment out the contents of a certain function, the program runs almost perfectly (actually I get some errors in that part, but I'm not sure if that is my fault or not). Well, if I don't comment out that function, I get an error. Turns out that function isn't even accessible! It's never even called. If I don't comment it out, but change the parameters of some unrelated function in some (also not called) function from 1 to 0, program runs. This is really bugging me and I'm starting to believe it's a problem with the compiler, or the EFS, because I don't understand what I could have done wrong.

Here's part of my main function:

Code:

int main()
{
int main()
{
    PA_Init();    // Initializes PA_Lib
    PA_InitVBL(); // Initializes a standard VBL
    
    PA_InitText(SCREEN_DOWN, 0);
    sprobj_InitspriteArray(); //Inicializes spriteArray. All SpriteObj pointers are now registered!
    bg_InitbgArray(); //Inicializes bgArray. All Background pointers are now registered!
    mc_InitmcArray();
    port_InitportArray();
    file_Initfilerecord();
    
    if (!EFS_Init(EFS_ONLY | EFS_DEFAULT_DEVICE, NULL))
        throwBadInitEFSException();
    
    Mainchar* mcDummy = NEW_Mainchar(ISPLAYER,
                SCREEN_UP,
                pol_TRDATA, //This is a const struct (of a defined struct type)
                0, 0, 0
                );

    Mainchar* mcTester = NEW_Mainchar(NOTPLAYER,
                SCREEN_UP, //screen
                dummy_TRDATA,
                64,64,0
                );

    ramfile* ftiles = file_open("bg_Tiles.bin");
    ramfile* fpal = file_open("bg_Pal.bin");

<more stuff>
}

The function file_open opens a file from the EFS filesystem. It usually works fine and I've opened lots of things with it, including those two files in the code. But, under the conditions I've explained before, when it tries to file_open("bg_Pal.bin"), it can't find the file. Again, it normally would find it (and of course it does exist).
I've also noticed that if I move the line "ramfile* fpal = file_open("bg_Pal.bin");" just below the EFS initialization, I get a undefined opcode error and the eulator crashes (if I move both file_open statements, the game... runs with no error (??!?))



Just in case you need more code, I'll put the code of the non-palib functions I use before the error pops out. I've triple-checked them, but you may check them too:

The array initialization functions (there shouldn't be any problem with those, but I'm kinda desperate)... I'll also put how I declared the arrays in each heade file. Also, they used to be pointers and the inits malloc'ed them. Doing it with pointers or arrays doesn't affect the problem, by the way.

Code:

void sprobj_InitspriteArray()
{
/*
spriteArray is declared like this in the header file:
#define SPROBJ_ARRAYSIZE 128
SpriteObj* spriteArray[2][SPROBJ_ARRAYSIZE];
u8 sprobj_amount[2];
*/
   PA_InitSpriteExtPrio(true);
   u8 j;
   for (j=0; j<SPROBJ_ARRAYSIZE; j++)
            (spriteArray[0])[j] = NULL;
   for (j=0; j<SPROBJ_ARRAYSIZE; j++)
            (spriteArray[1])[j] = NULL;
            
   sprobj_amount[SCREEN_DOWN] = 0;
   sprobj_amount[SCREEN_UP] = 0;
}

Code:

void bg_InitbgArray()
{
/*
bgArray declared as: Background* bgArray[2][4]; in header file
*/
s16 j;
for (j=0; j<4; j++)
(bgArray[0])[j] = NULL;
for (j=0; j<4; j++)
(bgArray[1])[j] = NULL;
}

Code:

void mc_InitmcArray()
{
/*
mcArray declared in header file as:
#define MC_ARRAYSIZE 120
u8 mc_amount;
Mainchar* mcArray[MC_ARRAYSIZE];
*/
u8 j;
for (j=0; j<MC_ARRAYSIZE; j++)
mcArray[j] = NULL;
mc_amount = 0;
}

Code:

void port_InitportArray(){
//portArray is declared in header file as: Portrait* portArray[4];
    u8 i;
    for (i=0; i<4; i++) portArray[i] = NULL;
}

Code:

void file_Initfilerecord()
{ /*filerecord is declared in the header file:
#define MAXFILES 50
ramfile* filerecord[MAXFILES];
/*
   u16 i;
   for (i=0; i< MAXFILES; i++)
   filerecord[i] = NULL;
}

The NEW_Mainchar function is quite long. Anyway it worked hundred of times.

Code:

Mainchar* NEW_Mainchar( bool isplayer,
u8 sscreen, //screen
const mc_TransformInfo info,
s16 mx, s16 my, s16 mz
 ) //map-relative coordinates
{
Mainchar* mc = (Mainchar*) malloc(sizeof(Mainchar));
if (mc != NULL)
{
s8 sprnum;
if (isplayer)
{
sprnum = 0;
mc->index = 0;
}
else { sprnum = sprobj_getslot(sscreen, 1, SPROBJ_ARRAYSIZE-1);
      if (sprnum == -1) throwSpriteFloodException();        //This just ouputs an error message, on screen and stops
mc->index = mc_getslot();
if (mc->index == -1) throwMaincharFloodException();
           }
if (info.shap == 1) throwUnsupportedShapeException();
mc->sprobj = NEW_SpriteObj (file_open(info.spr), sscreen, sprnum, info.pal,
info.shap, info.size, 0, 0, 2, 255);
mc->offset = info.offset;
mc->x = mx; mc->y = my; mc->z = mz;
mc->vx = 0; mc->vy = 0; mc->rvx = 0; mc->rvy = 0;

mc->acc  = info.acc;
mc->brak = info.brak;
        mc-> maxspeed = info.maxsp;

mc->speedzu = info.speedzu;
mc->speedzd = info.speedzd;
mc->jump = info.jump;
        mc->jumpsamount = info.jumpsamount;
mc->jumpangle = 0;
mc->height = info.height;
        mc->climbheight = info.climbheight;
        mc->climbheightdown = info.climbheightdown;
mc->jumpstore = mz;

mc->direction = DIR_DOWN;
mc->task = TSK_STAND;
mc->trans = info.trans;
copyanimdata(&(mc->anim), &(info.anim));

mc->radius = info.radius;
mc->center = info.center;

mc->jumpcount = 0;
mcArray[mc->index] = mc;
mc_amount++;
mc->avx = false; mc->avy = false;
mc->cvx = false; mc->cvy = false;

//properties
        mc->store = 0;
mc->onmove=info.onmove;
mc->solidness=info.solidness;
mc->mass=info.mass;
        
        //npc is null by default, has to be set later
        mc->npc = NULL;
}
else throwOutOfMemoryException();
return (mc);
}


SpriteObj* NEW_SpriteObj(ramfile* file, u8 sscreen, s8 sspritenum, u8 spalnum,
                        s32 sshape, s32 ssize, s16 sx, s16 sy, u8 zorderBg, u8 zorderSpr)
{
SpriteObj* s = NEW_SpriteObj_PRIVATE(file->fileloc, sscreen, sspritenum,spalnum,
                                     sshape, ssize, sx, sy, zorderBg, zorderSpr);
s -> spritefile = file;                                  
return (s);
}


SpriteObj* NEW_SpriteObj_PRIVATE(u8* sprdata, u8 sscreen, s8 sspritenum, u8 spalnum,
                        s32 sshape, s32 ssize, s16 sx, s16 sy, u8 zorderBg, u8 zorderSpr)
    {
    SpriteObj *sprobj = (SpriteObj*) malloc(sizeof(SpriteObj));
    if (sprobj != NULL)
        {
        sprobj->screen = sscreen;
        sprobj->spritenum = sspritenum;
        sprobj_loadpal(sscreen, spalnum);
        sprobj->palnum = spalnum;
        sprobj->x = sx; sprobj->y = sy; //screen-relative
        sprobj->shape = sshape; sprobj->size = ssize; //sprite width, height -> OBJ_SIZE-defined
        
        sprobj->sprite = (void*)sprdata;
        sprobj->animating = false;
        sprobj->frameQueue = NEW_IntCompCircQueue();
        sprobj->frame = 0;
        sprobj->zorderBg = zorderBg;
        sprobj->zorderSpr = zorderSpr;
        sprobj->visible = true; //visible by default.
        sprobj->hidden = false; //not hidden by default.
        
        
        PA_CreateSprite(sscreen, sspritenum, sprdata, sshape, ssize,
                               1, spalnum, sx, sy);
        PA_SetSpritePrio(sscreen, sspritenum, zorderBg);
        PA_SetSpriteExtPrio(sscreen, sspritenum, zorderBg);
        }
    else throwOutOfMemoryException();
    spriteArray[sscreen][sspritenum] = sprobj;
    (sprobj_amount[sscreen])++;
    return (sprobj);
    }

And finally the file_open function. This is gonna be long and messy since I gotta paste almost all the file module

Code:

ramfile* file_open (char* path)
{
   ramfile* f = file_findfile(path);
   if (f == NULL) f = file_load(path);
   else (f->users)++;

return (f);
}

ramfile* /*private*/ file_findfile(char* path)
{
      u16 i;
      ramfile* f = NULL;
      for(i = 0; i<MAXFILES; i++)
       if ((filerecord[i] != NULL) && !strcmp(path, filerecord[i]->path))
       {
f = filerecord[i];
break;
}
return (f);
}
      
ramfile* /*private*/ file_load (char* path)
{
 
   FILE* file = fopen(path, "r");
   ramfile* f = NULL;
   if (file != NULL)
        {
      fseek(file, 0, SEEK_END);
      size_t size = ftell(file);
      fseek(file, 0, SEEK_SET);
u8* buffer = (u8*)malloc(size * sizeof(u8*));
   if (buffer == NULL) throwOutOfMemoryException();
   fread(buffer, sizeof(u8), size, file);
  
   fclose(file);
   f = NEW_ramfile( size, buffer, path);
}
   else throwFileNotFoundException(path); ///////////////////////////////////////////////////THIS IS THE BRANCH THAT TRIGGERS THE EXCEPTION SO I KNOW WHY IT FAILED///////////////////////////////////////////////////
   return(f);
}

ramfile* /*private*/ NEW_ramfile (u64 size, u8* buffer, char* path)
{
   ramfile* f = malloc(sizeof(ramfile));
   if (f == NULL)
      throwOutOfMemoryException();
   else
      {
      f->fileloc = buffer;
      f->size = size;
f->path = malloc(sizeof(char)*strlen(path));
if ((f->path) == NULL) throwOutOfMemoryException();
strcpy(f->path, path);
      f->users = 1;
      u16 i = 0;
      while ((filerecord[i] != NULL) && (i<MAXFILES))
       i++;
      if (i>=MAXFILES) throwFileFloodException();
f->index = i;
filerecord[i] = f;
      }  
return(f);
}

I believe I don't access to memory that "doesn't belong to me", so I don't really know whose fault it is.

PLEASE HELP
 

[M. Lucanius]

Yikes that'll be a hard debug

Well, you say you get certain errors, but don't actually specify what kind. Are they compile errors or runtime errors? If compile errors, please post them as well. Otherwise, please describe what exactly goes wrong.

[Baro]

Yikes that's be a hard debug

Well, you say you get certain errors, but don't actually specify what kind. Are they compile errors or runtime errors? If compile errors, please post them as well. Otherwise, please describe what exactly goes wrong.

Runtime errors. When it attempts to open that file, it can't find it (even though it normally would). Other times, the errors are mallocs returning NULL. The weirdest thing is that it looks like it depends more on the amount of code compiled rather than its contents. And the fact that changing a constant in that other unrelated function from 1 to 0 makes a difference, hints that part of my code gets into parts of memory that belong to something else (not code).

[M. Lucanius]

hm... malloc returns null? well, at lest there it sounds like you're running out of memory. I have had the same (unresolved) problem before with a DS text editor I was working on (in fact it even hung on the free() function), but I was using FAT instead of EFS. If it worked before, you might try making sure you're not exceeding 4MB of RAM.

[Baro]

hm... malloc returns null? well, at lest there it sounds like you're running out of memory. I have had the same (unresolved) problem before with a DS text editor I was working on (in fact it even hung on the free() function), but I was using FAT instead of EFS. If it worked before, you might try making sure you're not exceeding 4MB of RAM.

Sometimes is the malloc, other times files that can't be found, but usually too early in the code to be actually running out of memory (before the main loop and not in any loop, so it's pretty easy to control the amount of memory used... also, the rom is currently only 534 kb), and probably because something overwrited whatever structure the NDS uses to manage allocated memory.

I forgot to say, that in some cases I ignored these errors and kept writing more code, and the error eventually disappears... only to appear when I write even more code.
This is so random it's crazy.

[M. Lucanius]

Wish I could help you out, but the problem seems rather illusive.

... And I must sleep now 

[t4ils]

Code:

#define SPROBJ_ARRAYSIZE 128
SpriteObj* spriteArray[2][SPROBJ_ARRAYSIZE];

What is the size of your SpriteObj structure ?

[Baro]

Code:

#define SPROBJ_ARRAYSIZE 128
SpriteObj* spriteArray[2][SPROBJ_ARRAYSIZE];

What is the size of your SpriteObj structure ?

Code:

typedef struct //SpriteObj
{ 
    ramfile* spritefile;
    s32   x, y; //screen-relative
    s32   shape, size; // OBJ_SIZE-defined
    IntCompCircQueue  *frameQueue; 
    void *sprite;
   
    u16 frame;
   
    s8 spritenum;
    u8 palnum;

    u8   screen;
    u8                zorderBg, zorderSpr;
   
   
    bool   animating;
    bool              visible; //Whether it's on screen or not.
    bool              hidden; //Not shown even if on screen
} SpriteObj;

According to sizeof, 40.

[t4ils]

Ok, it's not that big.
I tought maybe you have a variable too big that it goes past the 4MB.

I used to have this problem, not paying attention to my variables (some things like int array[5][256][192])

[Trenton_net]

You probably want to invest some time in writing a memory management class. You can avoid so many memory leaks and errors from doing it, while at the same time, it keeps your code that has access/touches memory clean and easy to maintain.

[Baro]

Ok, it's not that big.
I tought maybe you have a variable too big that it goes past the 4MB.

I used to have this problem, not paying attention to my variables (some things like int array[5][256][192])

Anyway, spriteArray is an array of pointers to SpriteObj, so the size of it is 4*2*128 which is 1024. I checked this, just in case.

I found out that pol_TRDATA and dummy_TRDATA (which are constants) are almost a kilobyte big (about 930 bytes). I've modified the functions that use them to accept pointers to the data instead of the data itself, making the main function 4kb smaller (counting that I used them two more times later on). This apparently made the things work again, but yet again, since small unrelated changes in the code made it apparently run, I'm not sure I've solved anything.
Would it make sense?

You probably want to invest some time in writing a memory management class. You can avoid so many memory leaks and errors from doing it, while at the same time, it keeps your code that has access/touches memory clean and easy to maintain.

How can I do this?

[Baro]

Anyway, spriteArray is an array of pointers to SpriteObj, so the size of it is 4*2*128 which is 1024. I checked this, just in case.

I found out that pol_TRDATA and dummy_TRDATA (which are constants) are almost a kilobyte big (about 930 bytes). I've modified the functions that use them to accept pointers to the data instead of the data itself, making the main function 4kb smaller (counting that I used them two more times later on). This apparently made the things work again, but yet again, since small unrelated changes in the code made it apparently run, I'm not sure I've solved anything.
Would it make sense?

It doesn't, since I'm still having this error.

Code:

void GUI_LoadStatIcons()
{
if (!GLOBAL_SHOWSTAMINA) GUI_UnLoadStatIcons();
if (  (GUImonitor.healthShown != global->health)
   || (GUImonitor.staminaShown != global->stamina)
   )
    {
    if (GUImonitor.fStatIcons == NULL)
        GUImonitor.fStatIcons = file_open("GUI/statIcons.Sprite");

    u8 healthIconNum  = (global->maxhealth > 32) ? 8
                      : ((global->maxhealth>>2) + 1);
    u8 staminaIconNum = !GLOBAL_SHOWSTAMINA ? 0
                      : (global->maxstamina > 32) ? 8
                      : ((global->maxstamina>>2) + 1);
<more stuff>

Right now the code causes a malloc error.

If I comment out the line GUImonitor.fStatIcons = file_open("GUI/statIcons.Sprite");, everything runs fine (the malloc error occurs AFTER it). This line was previously in another function, causing a malloc error too (If I put it again where it used to be, it works again, by the way). By that time, I tried moving that line along the code. Depending on where I placed it, the coude would: cause a malloc error, cause another file (existant, usually open without any problem) being unable to open, or no error. I think this proves it's not a memory management problem, I'm still with overlapping.

So I made a little debug function (fun fact: when I wrote it, the game worked again and would crash if I comment it out, so I had to write more code until I found the error again)

Code:

void DEBUG_testmem()
{
PA_InitText(SCREEN_DOWN, 0);
u64 n;
for (n=0; true; n++)
    {
    PA_OutputText(0,0,0, "%d", n*4);
    if (malloc(4) == NULL) PA_WaitFor(false); //guessing the minimum amount of memory you can reserve is 4
    }    
}

And tried it before and after the file_open("GUI/statIcons.Sprite"); line.
Results:
Before: 839172
After: 0

The size of GUI/statIcons.Sprite is 9Kb exactly.

It's either a problem in my file module or a bug in the EFS.

Code:

ramfile* file_open (char* path)
{
   ramfile* f = file_findfile(path);
   if (f == NULL) f = file_load(path);
   else (f->users)++;

return (f);
}

ramfile* /*private*/ file_findfile(char* path)
{
      u16 i;
      ramfile* f = NULL;
      for(i = 0; i<MAXFILES; i++)
       if ((filerecord[i] != NULL) && !strcmp(path, filerecord[i]->path))
       {
f = filerecord[i];
break;
}
return (f);
}
      
ramfile* /*private*/ file_load (char* path)
{
 
   FILE* file = fopen(path, "r");
   ramfile* f = NULL;
   if (file != NULL)
        {
      fseek(file, 0, SEEK_END);
      size_t size = ftell(file);
      fseek(file, 0, SEEK_SET);
u8* buffer = (u8*)malloc(size * sizeof(u8*));
   if (buffer == NULL) throwOutOfMemoryException();
   fread(buffer, sizeof(u8), size, file);
  
   fclose(file);
   f = NEW_ramfile( size, buffer, path);
}
   else throwFileNotFoundException(path);
   return(f);
}

ramfile* /*private*/ NEW_ramfile (u64 size, u8* buffer, char* path)
{
   ramfile* f = malloc(sizeof(ramfile));
   if (f == NULL)
      throwOutOfMemoryException();
   else
      {
      f->fileloc = buffer;
      f->size = size;
f->path = malloc(sizeof(char)*strlen(path));
if ((f->path) == NULL) throwOutOfMemoryException();
strcpy(f->path, path);
      f->users = 1;
      u16 i = 0;
      while ((filerecord[i] != NULL) && (i<MAXFILES))
       i++;
      if (i>=MAXFILES) throwFileFloodException();
f->index = i;
filerecord[i] = f;
      }  
return(f);
}

[Parmeisan]

I have been having exactly this sort of thing happening to me.  I encountered a problem where one part of my game would stop working - sometimes.  I went to try to narrow down the problem by commenting out large sections at a time, but it actually started working when I commented out anything at all, like even one line, and even a function that wasn't being called.  The most recent code I had added was using pointers improperly though, and when I fixed that, I didn't encounter the weird errors again for some time.  I recently did again, and am not seeing any pointer errors (though that doesn't mean there aren't any).  The thing is, my codebase is getting rather large to be posting it or asking anyone to comb through it, and I can't get the problem down to its basics because it is so utterly random-seeming.

Basically I'm just trying to say, I hear you man!  If you ever figure out for good what's causing yours, let us know, and I'll be sure to do the same.

[Baro]

Basically I'm just trying to say, I hear you man!  If you ever figure out for good what's causing yours, let us know, and I'll be sure to do the same.

Can't say if it's done until I see it doesn't come up again, but:

Code:

f->path = malloc(sizeof(char)*strlen(path));
if ((f->path) == NULL) throwOutOfMemoryException();
strcpy(f->path, path);

malloc(sizeof(char)*strlen(path)); should be malloc(sizeof(char)*strlen(path)+1);
and then f->path[strlen(path)] = 0;
Save space for an extra byte, and then set it to 0.