User avatar
jahboater
Posts: 8170
Joined: Wed Feb 04, 2015 6:38 pm
Location: Wonderful West Dorset

Re: advice about how to handle dynamic struct arrays (memset and others)

Tue Jan 04, 2022 11:21 am

dsyleixa123 wrote:
Tue Jan 04, 2022 11:11 am
memset(mvh + 200, 0, 2 * sizeof(MvHist));
i.e. what does the 200 and the 2 stand for :?:
Recall my original example above:

Code: Select all

MvHist *mvh = malloc( 200 * sizeof(MvHist) );
mvh = realloc( mvh, 202 * sizeof(MvHist) );
"mvh + 200" points to the start of the two new elements (realloc has simply increased the array size by two, by adding the two new elements at the end).
and "2" is obvious - you want to clear two elements!
dsyleixa123 wrote:
Tue Jan 04, 2022 11:11 am
( memset(mvh[pos], 0, sizeof(MvHist)); // does not work )
But this will work:

Code: Select all

memset(mvh + pos, 0, sizeof(MvHist));
"mvh" is a pointer to the first element of your array.
"mvh + 1" points to the second element.
"mvh + pos" points to "pos" elements past the start of the array.

mvh[pos] is completely different. It is exactly equivalent to "*(mvh + pos)"
Note the extra "*". You no longer have a pointer and of course memset() wont like that.

dsyleixa123
Posts: 1773
Joined: Mon Jun 11, 2018 11:22 am

Re: advice about how to handle dynamic struct arrays (memset and others)

Tue Jan 04, 2022 11:33 am

aaah, amazing, thanks a lot! :geek: :ugeek: 8-)

(I falsely even suspected that one had to add each time the sizeof(MvHist) to step from 1 array element to the next one, instead of just +1)
hobby programming retiree, ♂, GER

dsyleixa123
Posts: 1773
Joined: Mon Jun 11, 2018 11:22 am

Re: advice about how to handle dynamic struct arrays (memset and others)

Tue Jan 04, 2022 3:54 pm

moving other variables to the heap is easy and fine, especially also static arrays:

string filelist[128]; // edited
becomes
string *filelist = new string[128];

but making them dynamic like the mvh becomes rubbish (rather expectedly):
string *filelist = (string*)malloc( 128 * sizeof(string) );
Last edited by dsyleixa123 on Tue Jan 04, 2022 4:05 pm, edited 2 times in total.
hobby programming retiree, ♂, GER

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 31216
Joined: Sat Jul 30, 2011 7:41 pm

Re: advice about how to handle dynamic struct arrays (memset and others)

Tue Jan 04, 2022 4:01 pm

dsyleixa123 wrote:
Tue Jan 04, 2022 3:54 pm
moving other variables to the heap is easy and fine, especially also static arrays:

string filelist = new string[128];
becomes
string *filelist = new string[128];

but making them dynamic like the mvh becomes rubbish (rather expectedly):
string *filelist = (string*)malloc( 128 * sizeof(string) );
Something is awry here.

string filelist = new string[128]; <<< this probably wont compile
and
string *filelist = new string[128];

Are very different things, and are C++ statements. Note, that new will eventually use malloc or something similar under the skin.
Principal Software Engineer at Raspberry Pi Ltd.
Working in the Applications Team.

dsyleixa123
Posts: 1773
Joined: Mon Jun 11, 2018 11:22 am

Re: advice about how to handle dynamic struct arrays (memset and others)

Tue Jan 04, 2022 4:05 pm

yes, c+p mistake:

string filelist[128]; // edited
becomes
string *filelist = new string[128];

but how to make the size of the string array dynamic by malloc and realloc?
as stated,
string *filelist = (string*)malloc( 128 * sizeof(string) );
fails
hobby programming retiree, ♂, GER

User avatar
Paeryn
Posts: 3489
Joined: Wed Nov 23, 2011 1:10 am
Location: Sheffield, England

Re: advice about how to handle dynamic struct arrays (memset and others)

Wed Jan 05, 2022 3:26 am

dsyleixa123 wrote:
Tue Jan 04, 2022 4:05 pm
yes, c+p mistake:

string filelist[128]; // edited
becomes
string *filelist = new string[128];

but how to make the size of the string array dynamic by malloc and realloc?
as stated,
string *filelist = (string*)malloc( 128 * sizeof(string) );
fails
Is this string type C++'s std::string or a custom struct/class of your own?

If it's a simple struct then what you wrote with malloc/realloc works (you said the malloc failed but not what the failure was, the line as given is valid), if you use new/delete then you'd have to implement your own realloc function.

If it's C++'s std::string then reallocating the array may fail since realloc() will just do a blind copy if it needs to move the strings in the array (there may be pointers or other data in each string which could be invalidated by simply copying the memory).

Since you are using C++ then take advantage of std::vector which can be used like a dynamically sized array, it's more flexible but to just emulate an array something like:

Code: Select all

#include <string>
#include <vector>
#include <cstdio>

int main()
{
  // create the vector, pass a number to initialise that many elements
  std::vector<std::string> filelist(128); // start with 128 empty strings
  filelist[0] = "test_file";

  // get the number of elements
  printf("there are %d elements\n", filelist.size());

  // need to call c_str() on a string to get its underlying char* representation
  printf("filelist[0] = %s\n", filelist[0].c_str());
  
  // resize the vector
  filelist.resize(150); // now have 150 strings
  printf("now there are %d elements\n", filelist.size());
  
  // you can just add an element to the end and it will grow
  filelist.push_back("another_file");
  printf("now there are %d elements\n", filelist.size());
  
  // remove the last element and it will shrink
  filelist.pop_back();
  printf("and now there are %d again\n", filelist.size());
}
She who travels light — forgot something.
Please note that my name doesn't start with the @ character so can people please stop writing it as if it does!

dsyleixa123
Posts: 1773
Joined: Mon Jun 11, 2018 11:22 am

Re: advice about how to handle dynamic struct arrays (memset and others)

Wed Jan 05, 2022 8:40 am

the "string" is actually a Arduino C++ "String" class which is sort of stripped-down and modified std::string
It's used by Arduino IDE for the Zero and an ESP32.
I'll try the vector thing, hopefully it will work with Arduino and String, thanks for your advice!
hobby programming retiree, ♂, GER

dsyleixa123
Posts: 1773
Joined: Mon Jun 11, 2018 11:22 am

Re: advice about how to handle dynamic struct arrays (memset and others)

Wed Jan 05, 2022 8:50 am

update:
yes, it appearently works! 8-)

Code: Select all

void setup() {
   Serial.begin(115200);
   delay(1000);
   Serial.println();
   Serial.println("Serial() started");


   // create the vector, pass a number to initialise that many elements
   std::vector<String> filelist(128); // start with 128 empty Strings
   filelist[0] = "test_file";

   // get the number of elements
   printf("there are %d elements\n", filelist.size());

   // need to call c_str() on a String to get its underlying char* representation
   printf("filelist[0] = %s\n", filelist[0].c_str());

   // resize the vector
   filelist.resize(150); // now have 150 Strings
   printf("now there are %d elements\n", filelist.size());

   // you can just add an element to the end and it will grow
   filelist.push_back("another_file");
   printf("now there are %d elements\n", filelist.size());

   // remove the last element and it will shrink
   filelist.pop_back();
   printf("and now there are %d again\n", filelist.size());


}

Code: Select all

Serial() started
there are 128 elements
filelist[0] = test_file
now there are 150 elements
now there are 151 elements
and now there are 150 again
thank you very much!

BTW, is vector stored on Heap or on Stack?
hobby programming retiree, ♂, GER

Heater
Posts: 19105
Joined: Tue Jul 17, 2012 3:02 pm

Re: advice about how to handle dynamic struct arrays (memset and others)

Wed Jan 05, 2022 9:07 am

dsyleixa123 wrote:
Wed Jan 05, 2022 8:50 am
BTW, is vector stored on Heap or on Stack?
A vector is a structure (class) that contains:

A pointer to some buffer in which the data its stored.
A capacity - the size of that buffer.
A length - the number of elements of that buffer that are actually used.

That structure can exist on the stack if you declare a vector as a local variable. Or it can exist on the heap if you use new to create it.

Meanwhile, that buffer that holds your actual data is allocate don the heap. Which how a vector can. be resized, it just gets reallocated on the heap.

Edit: Actually instead of capacity the vector structure may contain a pointer to the end of the buffer. And instead of a length it may be a pointer to the end of the elements in use. But that is just an implementation detail.
Memory in C++ is a leaky abstraction .

dsyleixa123
Posts: 1773
Joined: Mon Jun 11, 2018 11:22 am

Re: advice about how to handle dynamic struct arrays (memset and others)

Wed Jan 05, 2022 9:41 am

thanks, heap is fine because I am short of free stack. 8-)
hobby programming retiree, ♂, GER

swampdog
Posts: 889
Joined: Fri Dec 04, 2015 11:22 am

Re: advice about how to handle dynamic struct arrays (memset and others)

Sun Jan 09, 2022 3:18 am

Vector method.

Personally I'd be replacing the "char" declarations in "MvHist" with a C++ string but as you mentioned arduino String (and I've never used one) I'll leave MvHist declared in C fashion using char[].

Just to show how easy it is to use std::vector<> to move and manipulate whole (or partial arrays)..

Code: Select all

#include <iostream>
#include <string>
#include <sstream>
#include <vector>
#include <cstring>

typedef struct {
        char smove[10];
        char piece[2];
}       MvHist;

typedef std::vector<MvHist> VecMH;

using std::cout;
using std::size_t;

void
DumpByIndex(const VecMH & vmh)
{
 for    (
        size_t i = 0;
        i < vmh.size();
        i++
        )       {

        cout
                <<"smove=\"" << vmh[i].smove << '"'
                << '\t'
                <<"piece=\"" << vmh[i].piece << '"'
                << '\n'
        ;
 }
}

void
DumpByIterator(const VecMH & vmh)
{
 for    (
        VecMH::const_iterator   i = vmh.begin();
        i != vmh.end();
        ++i
        )

        cout
                <<"smove=\"" << i->smove << '"'
                << ",\t"
                <<"piece=\"" << i->piece << '"'
                << '\n'
 ;
}

VecMH
CloneUpperCase(const VecMH & vmh)
{VecMH  t       (vmh);

 for    (
        size_t i = 0;
        i < t.size();
        i++
        )

        t[i].piece[0] = t[i].piece[0] - 'a' + 'A'
 ;

 return t;
}

void
CreateSomething(VecMH & r)
{
 for (size_t i=0; i<3; i++)     {
        MvHist                  mh;
        std::ostringstream      oss;

        oss << i;
        strncpy(mh.smove,oss.str().c_str(),sizeof mh.smove);
        mh.smove[sizeof mh.smove -1] = '\0';

        mh.piece[0] = (i % 26) +'a';
        mh.piece[sizeof mh.piece -1] = '\0';

        r.push_back(mh);
 }
}

void
DoAsC(const VecMH & vmh)
{const MvHist   *       mvp     (&vmh[0]);

 for (size_t i=0; i<vmh.size(); i++)
        printf("%s\t%s\n",mvp[i].smove,mvp[i].piece)
 ;
}

int
main()
{VecMH  vmh;

 cout << "size()=" << vmh.size() << '\n';
 CreateSomething(vmh);
 cout << "size()=" << vmh.size() << "\n\n";

 DumpByIndex(vmh);
 cout << '\n';
 DumpByIterator(vmh);
 cout << '\n';
 DumpByIndex(CloneUpperCase(vmh));
 cout << '\n';
 DoAsC(vmh);

 return 0;
}
I only included DumpByIterator() because it would be misleading to not mention iterators. However you don't need to know anything about them. Just throw the above code at g++ on an rpi.

dsyleixa123
Posts: 1773
Joined: Mon Jun 11, 2018 11:22 am

Re: advice about how to handle dynamic struct arrays (memset and others)

Sun Jan 09, 2022 9:23 am

yes, I admittedly don't absolutely understand the system of the nested shifting which is performed here, but thank you very much for your example!
hobby programming retiree, ♂, GER

User avatar
jahboater
Posts: 8170
Joined: Wed Feb 04, 2015 6:38 pm
Location: Wonderful West Dorset

Re: advice about how to handle dynamic struct arrays (memset and others)

Sun Jan 09, 2022 12:06 pm

dsyleixa123 wrote:
Sun Jan 09, 2022 9:23 am
the system of the nested shifting which is performed here
It is nothing to do with shifting.
The >> and << operators have been overloaded.
Its a popular way of doing I/O in C++.

dsyleixa123
Posts: 1773
Joined: Mon Jun 11, 2018 11:22 am

Re: advice about how to handle dynamic struct arrays (memset and others)

Sun Jan 09, 2022 1:22 pm

yes, you're right, wrong idiom, I meant "shifted" to the next expression, sorry for my poor English.
hobby programming retiree, ♂, GER

swampdog
Posts: 889
Joined: Fri Dec 04, 2015 11:22 am

Re: advice about how to handle dynamic struct arrays (memset and others)

Sun Jan 09, 2022 1:58 pm

Much of what C++ does in implemented via functions which are kind-of hidden via an "operator" function.

Code: Select all

#include <iostream>
#include <sstream>

int
main()
{std::string s("Hello World");

 std::cout << s << std::endl;

 std::cout.operator<<(std::istringstream(s).rdbuf());
 std::cout.operator<<(std::endl);

 return 0;
}
This is simpler to see..

Code: Select all

#include <string>
#include <cstdio>

struct foo {
std::string operator+(const std::string & s)
        {return std::string("Wierd")+s;}
};

int
main()
{foo f;
 std::string    s;

 s = f + "Stuff";
 std::printf("%s\n",s.c_str());

 s = f.operator+("Again");
 std::printf("%s\n",s.c_str());

 return 0;
}
..because I've defined in "foo" an "operator+()" method that's a close match for a "foo + std::string" then that's close enough for C++ to promote "Stuff" (a char[]) to std::string and the return type std::string matches thus the compiler can use that to perform this nonsensical addition.

There's a lot of these https://en.wikipedia.org/wiki/Operators ... nd_C%2B%2B. By convention "operator<<()" is used to insert data into a stream (ie: std::cout) and "operator>>()" is used to extract data from a stream (ie: std::cin). Effectively, cout=stdout, cin=stdin and cerr=stderr and like C, can be used on streams other than the terminal, typically (but not just limited to) files.

dsyleixa123
Posts: 1773
Joined: Mon Jun 11, 2018 11:22 am

Re: advice about how to handle dynamic struct arrays (memset and others)

Sun Jan 09, 2022 3:08 pm

yes, I already observed this stream thing several times, but I consider it as hard to read and to understand from 1st sight, kind of C++ obfuscation I actually do not really need (mostly) for my own purposes.
hobby programming retiree, ♂, GER

ABakhsh
Posts: 1
Joined: Mon Jun 20, 2022 10:22 am

Re: advice about how to handle dynamic struct arrays (memset and others)

Mon Jun 20, 2022 10:29 am

It would be better to use an image in order to achieve this result because it's supported by all major browsers. For example: https://codeprozone.com/code/css/38095/ ... order.html
Image

Return to “C/C++”