Development reference: Difference between revisions
No edit summary |
No edit summary |
||
(55 intermediate revisions by the same user not shown) | |||
Line 16: | Line 16: | ||
{| class="wikitable" | {| class="wikitable" | ||
! [http://www.aosabook.org/en/index.html Architecture of major open source apps] | ! [http://www.aosabook.org/en/index.html Architecture of major open source apps] | ||
|} | |||
{| class="wikitable" | |||
! [[Security]] | |||
|} | |} | ||
{| class="wikitable" | {| class="wikitable" | ||
Line 21: | Line 24: | ||
|} | |} | ||
{| class="mw-collapsible mw-collapsed wikitable" | {| class="mw-collapsible mw-collapsed wikitable" | ||
! | ! nosql enhances, not replaces, SQL | ||
|- | |- | ||
| | | Not all data should be denormalized, and not all data should be normalized. The optimal mix considers the extent of the data. | ||
* | |||
* | * Precise schemas are good when not overdone | ||
* | * When a container has an array with a large number of elements, it should be normalized | ||
* Sparse data and heterogeneous data are the best candidates for denormalization | |||
Postgres with JSON allows an elegant combination of nosql and SQL. | |||
|} | |} | ||
{| class="mw-collapsible mw-collapsed wikitable" | {| class="mw-collapsible mw-collapsed wikitable" | ||
Line 114: | Line 119: | ||
|} | |} | ||
{| class="mw-collapsible mw-collapsed wikitable" | {| class="mw-collapsible mw-collapsed wikitable" | ||
! | ! c++ in-memory storage of "major" objects | ||
|- | |||
| | |||
OBSERVATION ONE | |||
Consider An Important Qt Design: QObjects cannot normally be copied | |||
their copy constructors and assignment operators are private | |||
why? A Qt Object... | |||
might have a unique QObject::objectName(). If we copy a Qt Object, what name should we give the copy? | |||
has a location in an object hierarchy. If we copy a Qt Object, where should the copy be located? | |||
can be connected to other Qt Objects to emit signals to them or to receive signals emitted by them. If we copy a Qt Object, how should we transfer these connections to the copy? | |||
can have new properties added to it at runtime that are not declared in the C++ class. If we copy a Qt Object, should the copy include the properties that were added to the original? | |||
in other words, a QObject is a pretty serious object that has the ability to be tied to other objects and resources in ways that make copying dangerous | |||
isn't this true of all serious objects? pretty much | |||
OBSERVATION TWO | |||
if you have a vector of objects, you often want to track them individually outside the vector | |||
if you use a vector of pointers, you can move the object around much more cheaply, and not worry about costly large vector reallocations | |||
a vector of objects (not pointers) only makes sense if the number of objects is initially known and does not change over time | |||
OBSERVATION THREE | |||
STL vectors can store your pointers, iterate thru them, etc. | |||
for a vector of any substantial size, you want to keep objects sorted so you can find them quickly | |||
that's what my sorted_vector class is for; it simply bolts vector together with sort calls and a b_sorted status | |||
following STL practices, to get sorting, you have to provide operator< for whatever is in your vector | |||
BUT... you are not allowed to do operator<(const MyObjectPtr* right) because it would require a reference to a pointer which is not allowed | |||
BUT... you can provide a FUNCTOR to do the job, then provide it when sorting/searching | |||
a functor is basically a structure with a bool operator()(const MyObjectPtr* left, const MyObjectPtr* right) | |||
OBSERVATION FOUR | |||
unordered_set works even better when combining frequent CRUD with frequent lookups | |||
SUMMARY | |||
Dealing with tons of objects is par for the course in any significant app. | |||
Finding a needle in the haystack of those objects is also standard fare. | |||
Having multiple indices into those objects is also essential. | |||
Using unordered_set with object pointers and is very powerful. | |||
|} | |||
{| class="mw-collapsible mw-collapsed wikitable" | |||
! c++ stl reverse iterator skeleton | |||
|- | |- | ||
| | | From [http://www.sgi.com/tech/stl/ReverseIterator.html SGI]... | ||
reverse_iterator rfirst(V.end()); | |||
reverse_iterator rlast(V.begin()); | |||
while (rfirst != rlast) | |||
{ | |||
cout << *rfirst << endl; | |||
... | |||
rfirst++; | |||
} | |||
|} | |} | ||
{| class="mw-collapsible mw-collapsed wikitable" | {| class="mw-collapsible mw-collapsed wikitable" | ||
! | ! c++ stl reading a binary file into a string | ||
|- | |- | ||
| | | | ||
std::ifstream in("my.zip",std::ios::binary); | |||
if (!in) | |||
{ | |||
std::cout << "problem with file open" << std::endl; | |||
return 0; | |||
} | |||
in.seekg(0,std::ios::end); | |||
unsigned long length = in.tellg(); | |||
in.seekg(0,std::ios::beg); | |||
string str(length,0); | |||
std::copy( | |||
std::istreambuf_iterator< char >(in) , | |||
std::istreambuf_iterator< char >() , | |||
str.begin() | |||
); | |||
For more, see [[Reading a binary file|c++ stl reading a binary file]] | |||
|} | |||
{| class="mw-collapsible mw-collapsed wikitable" | |||
! C/C++ best-in-class tool selection | |||
|- | |||
| I need to have easy setup of debug-level tool support for portable C++11 code. And I need to decide and stick to it to be efficient. | |||
* '''Compiler selection''' | |||
** linux and mac: gcc | |||
** windows: Visual Studio | |||
* '''IDE selection''' | |||
** linux and mac: Qt Creator | |||
** windows: Qt Creator (OR Visual Studio OR eclipse?) | |||
* '''Debugger selection''' | |||
** linux and mac: Qt Creator | |||
** windows: Qt Creator (OR Visual Studio OR eclipse?) | |||
|} | |||
|} | |} | ||
<!-- | |||
=========================================================================================================================================================================================================================================================================================== | |||
--> | |||
{| class="mw-collapsible mw-collapsed wikitable" | |||
! c++11 | |||
|- | |||
| | |||
{| class="mw-collapsible mw-collapsed wikitable" | {| class="mw-collapsible mw-collapsed wikitable" | ||
! c++11 include base constructor (etc) in derived class | ! c++11 include base constructor (etc) in derived class | ||
Line 402: | Line 479: | ||
} STRING_PREF_INDEX; | } STRING_PREF_INDEX; | ||
|} | |} | ||
|} | |||
<!-- | |||
=========================================================================================================================================================================================================================================================================================== | |||
--> | |||
{| class="mw-collapsible mw-collapsed wikitable" | |||
! boost | |||
|- | |||
| | |||
{| class="mw-collapsible mw-collapsed wikitable" | |||
! boost release and debug build for linux | |||
|- | |||
| | |||
# download latest boost, eg: boost_1_59_0 | |||
m@wallee:~/development$ 7z x boost_1_59_0.7z | |||
rm boost && ln -s boost_1_59 boost | |||
cd boost | |||
build_boost_release_and_debug.sh | |||
# IMPORTANT: then patch .bashrc as instructed | |||
# cmake-###/clean.sh and cmake-###/build.sh are customized to match | |||
# (and older eclipse and server/nix/bootstrap.sh) | |||
To upgrade a CMake project: | |||
cd cmake-###/ | |||
./clean.sh | |||
./build.sh | |||
To upgrade an older autotools project: | |||
cd nix | |||
make distclean # removes nasty .deps folders that link to old boost if you let them | |||
make clean # removes .o files etc | |||
cd ../build-Release && make distclean && make clean | |||
cd ../build-Debug && make distclean && make clean | |||
cd .. | |||
./bootstrap force release | |||
./bootstrap force debug | |||
|} | |||
{| class="mw-collapsible mw-collapsed wikitable" | |||
! boost release and debug build for Windows | |||
|- | |||
| Open a VS2015 x64 Native Tools Command Prompt. | |||
EITHER: for new installs, you have to run bootstrap.bat first, it will build b2; | |||
OR: for reruns, remove boost dirs: [bin.v2, stage]. | |||
Then build 64-bit: | |||
cd "....\boost_1_59_0" | |||
b2 toolset=msvc variant=release,debug link=static address-model=64 | |||
rem trying to avoid excessive options, assuming I don't need these: threading=multi | |||
(old stuff) | |||
--toolset=msvc-14.0 address-model=64 --build-type=complete --stagedir=windows_lib\x64 stage | |||
Now open VS2013 x86 Native Tools Command Prompt and build 32-bit: | |||
cd "C:\Michael's Data\development\sixth_column\boost_1_55_0" | |||
bjam --toolset=msvc-12.0 address-model=32 --build-type=complete --stagedir=windows_lib\x86 stage | |||
|} | |||
{| class="mw-collapsible mw-collapsed wikitable" | {| class="mw-collapsible mw-collapsed wikitable" | ||
! c++ | ! boost regex (very similar to c++11, with LESS BUGS and MORE POWER) | ||
|- | |- | ||
| | | boost regex does everything you could need, take your time to get it right | ||
* regex_search will work hard to find substrings, while regex_match requires your regex to match entire input | |||
* smatch is a great way to get back a bunch of iterators to the results | |||
example: | |||
virtual bool url_upgrade_any_old_semver(string& url) | |||
{ | |||
// Perform a regex to update the embedded version if needed. | |||
// We have to see if (1) we have a semver and (2) it is not the current semver. | |||
// Only then do we take action. | |||
boost::regex regex("/([v0-9.]+?)/(.*)"); | |||
boost::smatch sm_res; | |||
if (boost::regex_match(url,sm_res,regex,boost::match_default)) | |||
{ | |||
string incoming_semver(sm_res[1].first,sm_res[1].second); | |||
if (incoming_semver != semanticVersion()) | |||
{ | |||
url = string("/")+semanticVersion()+"/"+string(sm_res[2].first,sm_res[2].second); | |||
return true; | |||
} | |||
} | |||
return false; | |||
} | |||
results: | |||
original string: string(sm_res[0].first,sm_res[0].second); | |||
first group: string(sm_res[1].first,sm_res[1].second); | |||
second group: string(sm_res[2].first,sm_res[2].second); | |||
etc. | |||
|} | |||
|} | |||
<!-- | |||
=========================================================================================================================================================================================================================================================================================== | |||
--> | |||
{| class="mw-collapsible mw-collapsed wikitable" | {| class="mw-collapsible mw-collapsed wikitable" | ||
! | ! C++ libraries | ||
|- | |- | ||
| | | | ||
{| class="mw-collapsible mw-collapsed wikitable" | {| class="mw-collapsible mw-collapsed wikitable" | ||
! | ! String escape formatting across different languages and systems | ||
|- | |- | ||
| | | | ||
* c++ to JSON: always use nlohmann::json j.dump() to encode, to ensure strings are properly escaped | |||
* JSON to c++: always use nlohmann::json j.parse() " | |||
* c++ to Javascript: use raw_to_Javascript() to properly escape | |||
* c++ to sqlite: use SqliteLocalModel::safestr(), which uses double_doublequotes(str) | |||
|} | |||
[[Postgres]] | |||
[[Simple-Web-Server]] | |||
[[Robot Operating System]] | |||
[[C++ https libraries]] | |||
[[Configure Qt development on Windows + Mac + linux]] | |||
[[Build the TagLib library with Visual Studio 2013]] | |||
|} | |} | ||
<!-- | |||
=========================================================================================================================================================================================================================================================================================== | |||
--> | |||
{| class="mw-collapsible mw-collapsed wikitable" | {| class="mw-collapsible mw-collapsed wikitable" | ||
! C/C++ | ! C/C++ building/linking | ||
|- | |- | ||
| | | | ||
{| class="mw-collapsible mw-collapsed wikitable" | {| class="mw-collapsible mw-collapsed wikitable" | ||
! | ! gcc makefile pain | ||
|- | |- | ||
| | | | ||
* I went through a LOT of pain to determine that gcc requires libraries to be listed AFTER the source and output parameters | |||
* set LD_LIBRARY_PATH to point to your libs if they are not in system location | |||
* Make sure Makefile uses TABS NOT SPACES. it's a FACT OF LIFE. hate it if you want. plenty more things to hate about linux/C as well. | |||
|} | |} | ||
{| class="mw-collapsible mw-collapsed wikitable" | {| class="mw-collapsible mw-collapsed wikitable" | ||
! gcc install multiple versions in ubuntu (4 and 5 in wily, eg) | ! gcc install multiple versions in ubuntu (4 and 5 in wily, eg) | ||
Line 559: | Line 657: | ||
--> | --> | ||
{| class=" | {| class="wikitable" | ||
! | ! [[git]] | ||
[[ | |||
|} | |} | ||
<!-- | <!-- | ||
Line 580: | Line 668: | ||
--> | --> | ||
{| class="mw-collapsible mw-collapsed wikitable" | {| class="mw-collapsible mw-collapsed wikitable" | ||
! | ! Debugging | ||
|- | |- | ||
| | | | ||
* | {| class="mw-collapsible mw-collapsed wikitable" | ||
* | ! Chrome capture large JSON variable | ||
|- | |||
| This is just pointlessly bizarre: | |||
* hit a breakpoint in the chrome debugger | |||
* right-click a variable and say "copy to global variable" (console will show name, typically "temp1") | |||
* push the variable to the clipboard by typing this in the console: | |||
copy(temp1) | |||
|} | |||
{| class="mw-collapsible mw-collapsed wikitable" | |||
! Visual Studio Code capture large string variable | |||
|- | |||
| While debugging, you can use the Debug Console to print memory, including the content of strings that are clipped by default in the variables and watch windows. | |||
View > Open View > Debug Console | |||
From there, send gdb a command to print memory – 300 characters of a string in this example: | |||
-exec x/300sb Query.c_str() | |||
|} | |} | ||
{| class="mw-collapsible mw-collapsed wikitable" | {| class="mw-collapsible mw-collapsed wikitable" | ||
! | ! Qt Creator conditional breakpoint | ||
|- | |- | ||
| | | To break on line number with string condition: | ||
* Set the breakpoint on the line as usual: F9 or click in left border | |||
* In the breakpoints list, rclick Edit | |||
* Add a condition; to break on a string value, THIS WORKS: | |||
((int)strcmp(strSymbol.c_str(), "GMO")) == 0 | |||
* BUT THERE IS A PRICE TO PAY. qt/gcc won't continue properly, unless you sset ANOTHER common breakpoint, hit it, and resume from that. It appears gcc continues (we hit the breakpoint after all), but qt does no logging or debugging, bah | |||
|} | |||
{| class="mw-collapsible mw-collapsed wikitable" | {| class="mw-collapsible mw-collapsed wikitable" | ||
! Qt Creator | ! Qt Creator and linux debug libraries | ||
|- | |- | ||
| | | | ||
Line 610: | Line 712: | ||
-- Up-to-date: /usr/local/include/libwebsockets.h | -- Up-to-date: /usr/local/include/libwebsockets.h | ||
(etc) | (etc) | ||
* Signals: At some point I needed to force gdb to pass SIGINT, SIGPIPE... signals to the app instead of gdb, after ssl kept causing Qt to pop a message and break in the debugger. To do so: | * Signals: At some point I needed to force gdb to pass SIGINT, SIGPIPE... signals to the app instead of gdb, after ssl kept causing Qt to pop a message and break in the debugger. To do so: | ||
Qt Creator->Options->Debugger-> | Qt Creator->Options->Debugger-> | ||
Line 622: | Line 723: | ||
at first i thought this would work: /usr/local/lib | at first i thought this would work: /usr/local/lib | ||
but i did a static build so i needed teh actual exe! COOL | but i did a static build so i needed teh actual exe! COOL | ||
|} | |||
{| class="mw-collapsible mw-collapsed wikitable" | |||
! QT Creator valgrind EASY | |||
|- | |||
| Valgrind is easy to use. Just find debugger pane top-left dropdown, switch mode from Debugger to Memcheck, and restart debugger. | |||
|} | |||
{| class="mw-collapsible mw-collapsed wikitable" | |||
! c/c++ gdb debugging | |||
|- | |||
| | |||
(gdb) help break | |||
Set breakpoint at specified line or function. | |||
Argument may be line number, function name, or "*" and an address. | |||
If line number is specified, break at start of code for that line. | |||
If function is specified, break at start of code for that function. | |||
If an address is specified, break at that exact address. | |||
With no arg, uses current execution address of selected stack frame. | |||
This is useful for breaking on return to a stack frame. | |||
Multiple breakpoints at one place are permitted, and useful if conditional. | |||
Do "help breakpoints" for info on other commands dealing with breakpoints. | |||
|- | |||
| ddd gives you a front end. I need to use it more, compare to other options | |||
|} | |} | ||
|} | |} | ||
Line 679: | Line 804: | ||
--> | --> | ||
{| class=" | {| class="wikitable" | ||
! | ! [[Javascript]] | ||
|- | |}<!-- | ||
{| class=" | |||
! | =========================================================================================================================================================================================================================================================================================== | ||
| | |||
--> | |||
{| class="wikitable" | |||
! [[Node.js]] | |||
|}<!-- | |||
=========================================================================================================================================================================================================================================================================================== | |||
--> | |||
{| class="wikitable" | |||
! [[React]] | |||
|}<!-- | |||
=========================================================================================================================================================================================================================================================================================== | |||
--> | |||
{| class="wikitable" | |||
! [[Vite]] | |||
|} | |} | ||
<!-- | |||
=========================================================================================================================================================================================================================================================================================== | |||
--> | |||
{| class="wikitable" | |||
! [[JSON]] | |||
|} | |} | ||
<!-- | <!-- | ||
Line 883: | Line 851: | ||
--> | --> | ||
{| class=" | {| class="wikitable" | ||
! | ! [[HTML]] | ||
|} | |} | ||
<!-- | |||
=========================================================================================================================================================================================================================================================================================== | |||
--> | |||
{| class="wikitable" | |||
! [[CSS]] | |||
|} | |} | ||
<!-- | |||
Line 960: | Line 872: | ||
--> | --> | ||
{| class="mw-collapsible mw-collapsed wikitable" | {| class="mw-collapsible mw-collapsed wikitable" | ||
! | ! SQL | ||
|- | |- | ||
| | | | ||
{| class="mw-collapsible mw-collapsed wikitable" | {| class="mw-collapsible mw-collapsed wikitable" | ||
! | ! Count records within a range | ||
|- | |- | ||
| | | This groups records into ranges, sorts by them, and gives a count, sweet: | ||
* | select count(*), id/1000000 as groupid from AccountHistory group by groupid; | ||
|} | |} | ||
[[postgres]] - [[sqlite]] - [[mysql]] - [[SQL Server]] - [[Robo 3T]] - [[DBeaver]] - [[pgadmin4]] | |||
|} | |} | ||
<!-- | <!-- | ||
Line 978: | Line 890: | ||
--> | --> | ||
{| class=" | {| class="wikitable" | ||
! | ! [[Meteor]] | ||
|} | |||
| | <!-- | ||
=========================================================================================================================================================================================================================================================================================== | |||
--> | |||
{| class="wikitable" | |||
! [[Android]] | |||
|} | |} | ||
<!-- | |||
=========================================================================================================================================================================================================================================================================================== | |||
--> | |||
{| class="wikitable" | |||
! [[Arduino]] | |||
|} | |} | ||
<!-- | |||
=========================================================================================================================================================================================================================================================================================== | |||
--> | |||
{| class="wikitable" | |||
! [[Raspberry Pi]] | |||
|} | |} | ||
<!-- | |||
=========================================================================================================================================================================================================================================================================================== | |||
--> | |||
{| class="wikitable" | |||
{| class=" | ! [[iOS]] | ||
! | |||
|} | |} | ||
<!-- | |||
=========================================================================================================================================================================================================================================================================================== | |||
--> | |||
{| class="wikitable" | |||
! [[.NET Core]] | |||
|} | |} | ||
<!-- | <!-- | ||
Line 1,081: | Line 948: | ||
--> | --> | ||
{| class="wikitable" | {| class="wikitable" | ||
! [[ | ! [[Java]] | ||
|} | |} | ||
<!-- | |||
=========================================================================================================================================================================================================================================================================================== | |||
--> | |||
{| class="wikitable" | {| class="wikitable" | ||
! [[ | ! [[Kotlin]] | ||
|} | |} | ||
<!-- | <!-- | ||
Line 1,159: | Line 968: | ||
--> | --> | ||
{| class=" | {| class="wikitable" | ||
! | ! [[Maven]] | ||
|} | |} | ||
<!-- | <!-- | ||
Line 1,182: | Line 978: | ||
--> | --> | ||
{| class=" | {| class="wikitable" | ||
! | ! [[Scala]] | ||
|} | |} | ||
<!-- | <!-- | ||
Line 1,201: | Line 989: | ||
--> | --> | ||
{| class="wikitable" | {| class="wikitable" | ||
! [[ | ! [[Python]] | ||
|} | |} | ||
<!-- | <!-- | ||
Line 1,211: | Line 999: | ||
--> | --> | ||
{| class="wikitable" | {| class="wikitable" | ||
! [[ | ! [[Go]] | ||
|} | |} | ||
<!-- | <!-- | ||
Line 1,245: | Line 1,033: | ||
--> | --> | ||
{| class="wikitable" | {| class="wikitable" | ||
! [[ | ! [[Bash basics]] but please prefer node or python :-) | ||
|} | |} | ||
<!-- | <!-- |
Latest revision as of 23:15, 25 November 2023
Design, programming and version control.
Patterns | ||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
C++ | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
|
c++11 | |||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
boost | ||||||
---|---|---|---|---|---|---|
|
C++ libraries | ||
---|---|---|
|
C/C++ building/linking | ||||
---|---|---|---|---|
|
git |
---|
Debugging | |||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
C | ||||
---|---|---|---|---|
|
Javascript |
---|
Node.js |
---|
React |
---|
Vite |
---|
JSON |
---|
HTML |
---|
CSS |
---|
SQL | ||
---|---|---|
postgres - sqlite - mysql - SQL Server - Robo 3T - DBeaver - pgadmin4 |
Meteor |
---|
Android |
---|
Arduino |
---|
Raspberry Pi |
---|
iOS |
---|
.NET Core |
---|
Java |
---|
Kotlin |
---|
Maven |
---|
Scala |
---|
Python |
---|
Go |
---|
PHP | |||
---|---|---|---|
|
Bash basics but please prefer node or python :-) |
---|
misc | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
|