FEDEVEL
Platform forum

kyfion , 09-22-2025, 01:34 PM

Hello all !

I've recently and for the first time created my own dbLib with access database and specific symbol and footprint for all my components. I used github to store the library and the components but the access file got about 800MB size which is too much for GitHub to handle.

Are there any suggestions for version controlling the access database ? Should I go with an other database since more people are going to work with this library ?
I would like to avoid an online db if possible because it get's laggy when searching components in big tables like capacitors for example.

Any suggestions or direction would be appreciated !!

Kind regards,
kyf

Robert Feranec , 09-23-2025, 02:29 AM

I am also curious how other people do this. Some time ago I tried to find prefect solution for local libraries ... but each solution had advantages and dissadvantages, I could not find a prefect one. Actually, altium cloud libraries are good .. but ... I prefer local solution. So if there are any ideas, I am very curious too.

QDrives , 09-24-2025, 02:48 PM

800MB for the database?
Version control on the database itself or data inside the database? How much guarantee do you need on it?
If you have the money, there is the 'on-premise' online libraries for Altium.

kyfion , 09-25-2025, 08:25 AM

Thank you @Robert Feranec and @QDrives for your replies.

I have my symbols, footprints and step files in separate files for each component (no duplicates) and I'm using MS Access for DB to connect all this information. The DB itself is too big 800MB. I'm not an experienced user with MS Access and I don't know if i messed up something to end with such huge DB.

The components that I have at the moment about 188000 components.

The thing is that the only way that I can imagine of a local DB lib getting version control is using CSV files instead of a DB because they are text files and they can perfectly work with version control such as git.

What I don't know is what are the limitations or the features that will miss with working with CSV files instead of MS access or SQL DBs.

I will try to convert my DB in CSV files and try that way in order to test it. I will let you know the results.

QDrives , 09-25-2025, 07:59 PM

So it would be around 4.2kB per component.
Have you tried "compacting" your database?
You can put a binary file in version control like Git, it is just all or nothing then.
Alternatively, you split your database into smaller parts, like resistors, capacitors, etc. That way, the all or nothing is limited to that type component.

kyfion , 09-26-2025, 02:09 PM

Hello. I've came up with an idea which I've tested and it seems that it is working. The method is quite simple.

I've converted my DB into CSV files. One CSV file for each DB table. Since CSV is text file it can be version controlled and also use tools like diff-merge to find out the diffs between files.

I've also created a script in python which converts the CSV files from a folder into an .xlsx file that contains all the CSV files as Sheets of the xlsx file. This is going to be used in the AD dbLib file as DB.

The total size from 800MB came down to 40MB and I haven't lost anything.

So the schema of creating a component ends up like this:

kyfion , 09-26-2025, 02:12 PM

kyfion , 09-26-2025, 02:14 PM

I would appreciate some feedback on this thought. Do you find it complicated ? Do you believe that there is any bugs hiding somewhere ?

Currently I've been using AD 25.7.1

kyfion , 09-26-2025, 02:15 PM

I have already my DB split into smaller parts as you mentioned QDrives.