conclusion
tidy up
PCB layout with shunt capacitor
KiCad files, like source code files, can be tracked in a Git repository. KiCad files are to some degree human readable, because they are just text. However, KiCad is not much like a text editor. It doesn't allow the user to directly edit the files; it provides an abstracted, graphical interface. It also does things that code editors do not do, such as come packaged with content, namely symbols and footprints. Even more, KiCad has a notion of a library in which symbols and footprints are contained. In these respects, KiCad more resembles an IDE than a text editor. But it is an IDE that does not provide an interface to version control.
The metaphor is rough and not all that useful. The ultimate question is how can the data of a KiCad project be managed in version control similarly to a software project?
Perhaps you'd want to know why that would be desirable. It is because software projects have evolved to facilitate very useful things.
Reproducibility: the recipient of a copy of the project has exactly the same data as the sender, which the recipient's tool processes to create exactly the same artifacts that the sender's tool produces. This, in turn, facilitates collaboration, as there is universal agreement about the content of the project. Alice and Bob only need to make sure they are working against the same version of the project and using versions of the tool that do not introduce breaking changes.
Another thing we'd like to do is apply this same version control idea to dependencies, which allows projects to incorporate other work and consequently achieve more ambitious aims. KiCad also lacks the dependency control that is now common among programming language tooling.
Understanding how to make KiCad work like this requires knowing what it does.
Hence the reason for this repository, which is my attempt at learning precisely that.
A prompt appears, stating that the directory is not empty. It asks if I want to put the project in its own directory, as recommended. Click "No".
KiCad creates three files:
You want to draw a schematic now. A schematic requires symbols.
A symbol is data contained within a symbol library file. KiCad supports several library file formats.
KiCad is packaged with symbol libraries. On my machine, they are installed in /usr/share/kicad/library/. The included libraries are organized by the "type" of component; for instance, there is a 4xxx.lib for 4000-series logic ICs, and a Analog_DAC.lib for digital to analog converter ICs. Some libraries refer to manufacturers and lines of products, such as NXP Kinetis microcontrollers.
The built-in libraries have global scope: every KiCad project on the computer may refer to them. KiCad manages the set of global libraries in a file, which on my computer has the path $HOME/.config/kicad/sym-lib-table. This file lists the name and path of each library, plus other information.
Importantly, library paths may contain variables. KiCad defines two variables: KICAD_SYMBOL_DIR, which points at the global libraries, and KIPRJMOD which points to the project directory, for project-specific libraries. Library paths need not refer to these variables; absolute paths are also valid. For instance, third-party library collections on my system (from SnapEDA and Digi-Key) are referred by absolute paths.
The obvious advantage of the built-in libraries is their coverage of a large number of popular components. This makes it possible to begin drawing a schematic immediately, without having to source symbol libraries yourself.
It's almost like the standard library of a programming language. It might come packaged with the compiler and the rest of the toolchain. Good standard libraries have very stable APIs: they rarely or never introduce breaking changes, and users are given ample advance warning of breaking changes. Standard libraries vary in scope among languages; C's is minimal, Go's is extensive but delimited.
Whether KiCad built-in symbols have comparable quality to good standard libraries, I don't know. By comparison, it seems odd to me that KiCad would come with any symbols that are specific to manufacturers; in such a software-inspired universe, it would be another entity's responsibility (a manufacturer or community) to produce, maintain, package, and distribute these symbols. Users could use Git submodules or other tooling to manage dependencies. Versioning the libraries puts users in direct control of the content of their project, even though that content is distributed. Users upgrade a dependency per their own initiative.
It's clear, from this discussion, that global libraries are not useful in a collaborative, dependency-managing context. They are outside of the KiCad project's directory and thus are not tracked by version control. Since they are shipped with KiCad itself, they can vary across KiCad versions, risking inconsistency and breakage.
KiCad does something interesting, however. It caches symbols that the project uses in a file local to the project. Let's observe this.
Open Eeschema, type a, then r, hit enter, and click to place a resistor.
Save the schematic.
KiCad touches the following files:
learn-git-kicad.sch is modified to contain the placed resistor. Sheet data is also filled in; effectively it has made a sheet with a resistor.
learn-git-kicad-cache.lib is created. This is the cached symbol library!
learn-git-kicad.sch-bak is created. This is a backup of the previous version of the schematic, which was empty. Completely unnecessary, when using version control, because the previous version is already committed. Git should be instructed to ignore this.
We can take the symbol cache and turn it into a project-specific library, so that the project no longer uses the global source. This project-specific library would be checked in to version control, so all collaborators will get the same data, regardless of what KiCad version they use. Good. How?
KiCad touched the following
However, this has not changed anything about the schematic. In particular, the resistor still refers to a global library!
To fix this, we have to point the resistor to the project-specific library.
e.KiCad touched the following:
learn-git-kicad.sch was modified to refer to the resistor symbol in the project-specific library.
learn-git-kicad-cache.lib was modified to prepend the library name to the symbol names.
At this point, I'm highly confident that collaborators will get exactly the same schematic that I share with them.
Now I want to do a PCB layout, which means I have to assign footprints to all the symbols that need them. Before this, we have to annotate the schematic. KiCad changes the schematic file.
Much like with symbols, KiCad also comes with footprints. Similarly, they come in libraries which can be global or project-specific. Their paths are specified in table files, as with symbol libraries. But there are three variables that may appear in footprint library paths:
It so happens that footprints might be less generic than symbols. Particularly, connectors such as TRS jacks, and things like potentiometers have no standardized footprints, and the built-in libraries do not contain every footprint in existence. So while the built-in libraries may be useful for common parts such as surface-mount resistors, it's unlikely they will suffice for all the parts in a design. This is all the more true for 3D models, which are not strictly necessary for PCB manufacturing, but aid mechanical design.
It's amazing that KiCad offers all of this functionality as an open-source project with free-to-download builds. If you're reading this, you deserve the satisfaction of donating to KiCad.
It is thus instructive to demonstrate how one might source an uncommon footprint and subject it to the same version control as we did the resistor symbol.
I've selected a 1% 1-kΩ resistor from Stackpole, part number
RNMF14FTC1K00
which is somewhat unusual for having a smaller footprint than ordinary 0.25-W resistors. KiCad has some footprints that look like they would work, but I'd feel better using something made specifically for this part.
First, go to the symbol list and add the MPN field. May as well fill in the datasheet field too.
Ultra Librarian provides two files of interest:
Move the .kicad_mod file into the project directory. Go to "Preferences", then "Manage Footprint Libraries". Click the "Project Specific Libraries" tab, and then the button to add an existing library. Choose the project directory. Click OK for both dialogues.
KiCad creates two files:
fp-info-cache is very large, seemingly containing a cache of info about footprints in global libraries. Seems like something to ignore?
fp-lib-table is akin to sym-lib-table: it specifies the path of our newly-added footprint library.
When you open Eeschema, it complains:
This schematic was made using older symbol libraries which may break the schematic. Some symbols may need to be linked to a different symbol name. Some symbols may need to be "rescued" (copied and renamed) into a new library.
What a confusing message. Older than what? What may cause breakage: the making of the schematic using older symbol libraries, or the older symbol libraries? Why would something being "older" cause breakage? And I don't understand anything about "linking" symbols to different names.
Basically, WTF?
We're using Git, so anything is reversible. I'll close the dialogue with OK and see what happens. KiCad claims it is going to
Rescue symbol learn-git-kicad-cache:Device_R found only in cache library to learn-git-kicad-rescue:Device_R-learn-git-kicad-cache.
Actually, that sounds like a terrible thing to do. Way to make names even more confusing than they already were. But I think I've learned something.
KiCad claims that the symbol is found only in the cache, not in any library. Wouldn't this be incorrect, given that the cache is a library? What library is it referring to? I would agree that the symbol isn't found in the global libraries, because it was renamed; the original cached version had no prefix, but the prefix was added after we used the symbol in the schematic!
I think KiCad has misunderstood what I've done. It thinks that my cache contains symbols from old, possibly outdated global KiCad libraries, and that I intend to continue using global libraries. I may or may not, but I don't see why anything has to change.
Even more strangely, there is a button in the lower left corner to "Never Show Again". No explanation of why anybody would want to do that!
What happened?
Per KiCad's documentation:
When loading a schematic created prior to the symbol library table implementation, Eeschema will attempt to remap the symbol library links in the schematic to the appropriate library table symbols.
Rescuing cached symbols
By default, Eeschema loads symbols from the project libraries according to the set paths and library order. This can cause a problem when loading a very old project: if the symbols in the library have changed or have been removed or the library no longer exists since they were used in the project, the ones in the project would be automatically replaced with the new versions. The new versions might not line up correctly or might be oriented differently leading to a broken schematic.
When a project is saved, a cache library with the contents of the current library symbols is saved along with the schematic. This allows the project to be distributed without the full libraries. If you load a project where symbols are present both in its cache and in the system libraries, Eeschema will scan the libraries for conflicts. Any conflicts found will be listed in the following dialog:
Rescue conflicts dialog
You can see in this example that the project originally used a diode with the cathode facing up, but the library now contains one with the cathode facing down. This change would break the schematic! Pressing OK here will cause the symbol cache library to be saved into a special ``rescue'' library and all the symbols are renamed to avoid naming conflicts.
If you press Cancel, no rescues will be made, so Eeschema will load all the new components by default. If you save the schematic at this point, your cache will be overwritten and the old symbols will not be recoverable. If you have saved the schematic, you can still go back and run the rescue function again by selecting "Rescue Cached Components" in the "Tools" menu to call up the rescue dialog again.
If you would prefer not to see this dialog, you can press "Never Show Again". The default will be to do nothing and allow the new components to be loaded. This option can be changed back in the Libraries preferences.
So essentially, the reason that KiCad caches symbols at all is because it is making up for a lack of version and dependency control that works in a distributed context. Instead of delegating this problem to an external tool, as might have been suggested by the Unix philosophy, KiCad decided to implement its own half-baked solution.
Per KiCad's documentation, caching is used to allow distribution without the full libraries. But the cache is a second-class citizen; it is never an actual library in itself, just a stand-in for the real libraries bundled with KiCad or provided by third parties. But oops, those real libraries might change quietly, flipping all your diodes around backwards, if your schematic is too old relative to the symbol libraries.
The problem isn't the age of the schematic, but the lack of user-controlled dependency management. Frankly, none of this should be KiCad's problem. KiCad should stick to editing symbols, schematics, footprints, and PCBs, and leave dependency management and version control to software that provides a full solution for it. It should remove support for global libraries and require all users to manage libraries themselves. Why? Because every project is different, and each project should have the ability to decide for itself what kind of dependency management is appropriate.
If a dependency is highly trusted not to cause breaking changes, only improvements, then a user should have the ability to let that dependency update automatically. (This is rarely the right choice.) If a dependency is less trusted, a user should be able to lock their project to a version of that dependency, preventing unanticipated breaking changes. If a dependency is so untrusted that it may not even exist online after a reasonable amount of time, the user should be able to "vendor" it; copy it into the project tree without any expectation of upstream bugfixes or improvements.
This allows us to load the schematic without any confusing messages!
We're finally ready to add footprints. It's easy. The only difference from symbol libraries is that a footprint library is a directory, where each footprint is a file.
From here, it's a matter of designing the PCB---trivial for a circuit which is just an open resistor! Then, we can submit the design to a PCB manufacturer to get a quote. I'm sure they'll respond quickly.
Take a moment to reflect. This Git repository can be cloned, and the resulting clone has zero dependence on the built-in libraries. This is something to celebrate.
/ \
(---)
> <
It's a glencairn glass with a beverage of your choice.
The engineering design process has many linear parts, but overall there are feedback loops that make "future" design phases entail changes in "prior" design phases. We've worked out an electrical design, but the finished product probably has to satisfy mechanical, supply chain, manufacturing, and cost constraints. Some of these constraints can change unpredictably.
But software engineering has evolved to deal with similar uncertainty, and I think its principles can be applied to electrical engineering.
PCB routing might change if it the board has to fit into an off-the-shelf enclosure.
Footprints might change, even without a component changing. Pads can be enlarged or shifted; thru-hole components can have their leads formed to either axial or radial configurations.
Symbols might change, in the event a microcontroller goes out of stock and has to be substituted with something with a different pinout.
Electrical design might change, which may require rerouting. Consider pin swaps in microcontrollers and unit swaps in multi-unit ICs.
Requirements might change, possibly entailing addition of new symbols and footprints.
The design might be changed to cut costs in large-scale manufacturing: minimizing the size of the bill of materials, maximizing the number of repeated parts.
The changes that seem most relevant to KiCad and Git would be those affecting symbols and footprints, as these have to do with files. Footprints are pretty easy, because each footprint is a separate file. Symbols, however, can be stored many to a single file.
For example, suppose we want to add a capacitor symbol from the built-in libraries. When we add this symbol to the schematic, it will appear in the cache. But there's no easy way to move it from the cache into our library.
A better way is to use the symbol editor.
Open the symbol editor. Find the capacitor symbol (Device:C) and right click it. Click "Save a copy as...". In the dialogue, select the library called "symbols". Click "Save".
The symbol is copied verbatim into the local library!
Now we can use that symbol in the schematic.
Easy. Just move the file into the project directory, which adds it to the footprint library.
Let's
At this point, I basically understand what's going on.
KiCad caches and backs up files because it's pretending to be Git. Ignore the caches and backups and check the real files into Git.
KiCad supports project-specific symbol and footprint libraries; this rocks. Because of this feature, it's possible for the user to decide the dependency management scheme that works for them, whether it is automatic updates, version locking, or vendoring. In addition, version locking can be achieved using Git submodules, reducing the need for third-party tooling.
For non-critical parts of the project, it's OK to use global libraries if you're lazy. I highly doubt that the bundled resistor and capacitor symbols are going to break my project. However, symbols for diodes, and footprints generally, seem to be a greater risk.