📚 Agora node [[architecture]] ☆
📜 File architecture.md by @flancian ☆ 📎 ️🔗 ✍️ ⛶
- I like [[christopher alexander]], [[lacaton and vassal]], [[solar punk]], [[arcologies]].
📜 File architecture.md by @communecter ☆ 📎 ⛶
Architecture§
following this suggestion everything is centered around a MongoDB Database
web instance§
Techno§
PHP framework Yii§
WE are still using php 5.6 php7 will be set up first semester 2021 We still need to switch to Yii 2 it’s in progress
javascriptJ§
we still use the good old Jquery et des centaines de pluggins web /pixelhumain/ph/plugins
MongoDB§
We chose a No SQL database for it’s simplicity et mutability our schemas evolve permenantly and almost dynamically the whole system is in the end à huge collection of well organised JSON documents We try to follow as much as possible common object standards from schema.org
hébergement§
OVH instance type de serveur
stockage BDD§
instance type de serveur
schema de BDD§
costum : personnalisde integrated instance§
json configs, specific designs
use CO and personnalize freely connect it to your own DNS show only the data of your community while continuing to contribute to a common Database
COmobi§
NodeJs, Meteor, CO.api, MongoDB https://play.google.com/store/apps/details?id=org.communecter.mobile&hl=en_US
Oceco Mobi§
NodeJs, Meteor, CO.api, MongoDB https://play.google.com/store/apps/details?id=org.communecter.oceco&hl=fr
Usage de systeme externe et interopérabilité§
nominatim > overpass API api-addresse.data.gouv.fr mediawiki gitlab
Features Technical documentation§
[COstum](./07 - COstum/costum.md)§
[CMS](./04 - Documentation technique/cmsEngine.md)§
[autre](./07 - COstum/CMS - Templates Blocks.md)
[COforms](./08 - COForms/coforms Tech.md)§
[SearchObj](./04 - Documentation technique/process/search.md)§
[mapObj](./04 - Documentation technique/process/mapObj.md)§
Hebergement des COTOOLS§
📜 File en/architecture.md by @communecter ☆ 📎 ⛶
Architecture§
following this suggestion everything is centered around a MongoDB Database
web instance§
Techno§
PHP framework Yii§
WE are still using php 5.6 php7 will be set up first semester 2021 We still need to switch to Yii 2 it’s in progress
javascriptJ§
we still use the good old Jquery et des centaines de pluggins web /pixelhumain/ph/plugins
MongoDB§
We chose a No SQL database for it’s simplicity et mutability our schemas evolve permenantly and almost dynamically the whole system is in the end à huge collection of well organised JSON documents We try to follow as much as possible common object standards from schema.org
hébergement§
OVH instance type de serveur
stockage BDD§
instance type de serveur
costum : personnalisde integrated instance§
json configs, specific designs
use CO and personnalize freely connect it to your own DNS show only the data of your community while continuing to contribute to a common Database
COmobi§
NodeJs, Meteor, CO.api, MongoDB https://play.google.com/store/apps/details?id=org.communecter.mobile&hl=en_US
Oceco Mobi§
NodeJs, Meteor, CO.api, MongoDB https://play.google.com/store/apps/details?id=org.communecter.oceco&hl=fr
Usage de systeme externe et interopérabilité§
nominatim > overpass API api-addresse.data.gouv.fr mediawiki gitlab
Features Technical documentation§
[COstum](./07 - COstum/costum.md)§
[CMS](./04 - Documentation technique/cmsEngine.md)§
[autre](./07 - COstum/CMS - Templates Blocks.md)
[COforms](./08 - COForms/coforms Tech.md)§
[SearchObj](./04 - Documentation technique/process/search.md)§
[mapObj](./04 - Documentation technique/process/mapObj.md)§
Hebergement des COTOOLS§
📜 File docs/advanced/architecture.md by @matthieuG ☆ 📎 ⛶
Quartz is a static site generator. How does it work?
This question is best answered by tracing what happens when a user (you!) runs npx quartz build in the command line:
On the server§
- After running
npx quartz build, npm will look atpackage.jsonto find thebinentry forquartzwhich points at./quartz/bootstrap-cli.mjs. - This file has a shebang
- Transpiling and bundling the rest of Quartz (which is in Typescript) to regular JavaScript using esbuild. The
esbuildconfiguration here is slightly special as it also handles.scssfile imports using esbuild-sass-plugin v2. Additionally, we bundle ‘inline’ client-side scripts (any.inline.tsfile) that components declare using a customesbuildplugin that runs another instance ofesbuildwhich bundles for the browser instead ofnode. Modules of both types are imported as plain text. -
Running the local preview server if
--serveis set. This starts two servers:- A WebSocket server on port 3001 to handle hot-reload signals. This tracks all inbound connections and sends a ‘rebuild’ message a server-side change is detected (either content or configuration).
- An HTTP file-server on a user defined port (normally 8080) to serve the actual website files.
- If the
--serveflag is set, it also starts a file watcher to detect source-code changes (e.g. anything that is.ts,.tsx,.scss, or packager files). On a change, we rebuild the module (step 2 above) using esbuild’s rebuild API which drastically reduces the build times. - After transpiling the main Quartz build module (
quartz/build.ts), we write it to a cache file.quartz-cache/transpiled-build.mjsand then dynamically import this usingawait import(cacheFile). However, we need to be pretty smart about how to bust Node’s import cache so we add a random query string to fake Node into thinking it’s a new module. This does, however, cause memory leaks so we just hope that the user doesn’t hot-reload their configuration too many times in a single session :)) (it leaks about ~350kB memory on each reload). After importing the module, we then invoke it, passing in the command line arguments we parsed earlier along with a callback function to signal the client to refresh.
build.ts, we start by installing source map support manually to account for the query string cache busting hack we introduced earlier. Then, we start processing content:- Clean the output directory.
- Recursively glob all files in the
contentfolder, respecting the.gitignore. -
Parse the Markdown files.
- Quartz detects the number of threads available and chooses to spawn worker threads if there are >128 pieces of content to parse (rough heuristic). If it needs to spawn workers, it will invoke esbuild again to transpile the worker script
quartz/worker.ts. Then, a work-stealing workerpool is then created and batches of 128 files are assigned to workers. - Each worker (or just the main thread if there is no concurrency) creates a unified parser based off of the plugins defined in the [[configuration]].
-
Parsing has three steps:
- Read the file into a vfile.
- Applied plugin-defined text transformations over the content.
- Slugify the file path and store it in the data for the file. See the page on [[paths]] for more details about how path logic works in Quartz (spoiler: its complicated).
- Markdown parsing using remark-parse (text to mdast).
- Apply plugin-defined Markdown-to-Markdown transformations.
- Convert Markdown into HTML using remark-rehype (mdast to hast).
- Apply plugin-defined HTML-to-HTML transformations.
- Quartz detects the number of threads available and chooses to spawn worker threads if there are >128 pieces of content to parse (rough heuristic). If it needs to spawn workers, it will invoke esbuild again to transpile the worker script
- Filter out unwanted content using plugins.
-
Emit files using plugins.
- Gather all the static resources (e.g. external CSS, JS modules, etc.) each emitter plugin declares.
-
Emitters that emit HTML files do a bit of extra work here as they need to transform the hast produced in the parse step to JSX. This is done using hast-util-to-jsx-runtime with the Preact runtime. Finally, the JSX is rendered to HTML using preact-render-to-string which statically renders the JSX to HTML (i.e. doesn’t care about
useState,useEffect, or any other React/Preact interactive bits). Here, we also do a bunch of fun stuff like assemble the page [[layout]] fromquartz.layout.ts, assemble all the inline scripts that actually get shipped to the client, and all the transpiled styles. The bulk of this logic can be found inquartz/components/renderPage.tsx. Other fun things of note:- CSS is minified and transformed using Lightning CSS to add vendor prefixes and do syntax lowering.
- Scripts are split into
beforeDOMLoadedandafterDOMLoadedand are inserted in the<head>and “ respectively.
- Finally, each emitter plugin is responsible for emitting and writing it’s own emitted files to disk.
- If the
--serveflag was detected, we also set up another file watcher to detect content changes (only.mdfiles). We keep a content map that tracks the parsed AST and plugin data for each slug and update this on file changes. Newly added or modified paths are rebuilt and added to the content map. Then, all the filters and emitters are run over the resulting content map. This file watcher is debounced with a threshold of 250ms. On success, we send a client refresh signal using the passed in callback function.
On the client§
- The browser opens a Quartz page and loads the HTML. The
<head>also links to page styles (emitted topublic/index.css) and page-critical JS (emitted topublic/prescript.js) - Then, once the body is loaded, the browser loads the non-critical JS (emitted to
public/postscript.js) -
Once the page is done loading, the page will then dispatch a custom synthetic browser event
"nav". This is used so client-side scripts declared by components can ‘setup’ anything that requires access to the page DOM.- If the [[enableSPA option]] is enabled in the [[configuration]], this
"nav"event is also fired on any client-navigation to allow for components to unregister and reregister any event handlers and state. - If it’s not, we wire up the
"nav"event to just be fired a single time after page load to allow for consistency across how state is setup across both SPA and non-SPA contexts.
- If the [[enableSPA option]] is enabled in the [[configuration]], this
The architecture and design of the plugin system was intentionally left pretty vague here as this is described in much more depth in the guide on [[making your own plugin]].
📜 File architecture.md by @agora@botsin.space ☆ 📎 ⛶
✨ AI Synthesis Mistral Gemini ChatGPT Claude
Expanding this section will automatically generate an AI synthesis of the contributions in this node.
