From d2ed8585efa444eb8c2814f7d024154e446eb244 Mon Sep 17 00:00:00 2001 From: nyg Date: Sun, 14 May 2023 14:04:37 +0200 Subject: [PATCH] Add jdbc and log4j code examples --- docs/index.md | 12 ++ docs/prog/apollo-graphql.md | 4 +- .../java-io-streams.md => java/io-streams.md} | 4 +- docs/prog/java/jdbc.md | 128 ++++++++++++++++++ docs/prog/java/log4j2.md | 95 +++++++++++++ ...on_mutuelle.md => 2-exclusion-mutuelle.md} | 5 +- docs/prog/pco/{3_threads.md => 4-threads.md} | 5 +- ..._semaphores.md => 5-verrous-semaphores.md} | 5 +- ...eurs.md => 6-producteurs-consommateurs.md} | 5 +- .../pco/{6_moniteurs.md => 7-moniteurs.md} | Bin 2202 -> 2208 bytes ...redacteurs.md => 8-lecteurs-redacteurs.md} | 4 +- docs/prog/pco/{summary.md => resume.md} | 0 ...on-cheat-sheet.md => python-cheatsheet.md} | 4 +- docs/prog/res/smtp.md | 4 +- docs/prog/res/udp.md | 5 +- docs/prog/resume-tweb.md | 2 +- docs/prog/sagemath.md | 6 +- docs/softwares/maven.md | 2 +- mkdocs.yml | 31 +++-- 19 files changed, 287 insertions(+), 34 deletions(-) rename docs/prog/{res/java-io-streams.md => java/io-streams.md} (99%) create mode 100644 docs/prog/java/jdbc.md create mode 100644 docs/prog/java/log4j2.md rename docs/prog/pco/{1_exclusion_mutuelle.md => 2-exclusion-mutuelle.md} (97%) rename docs/prog/pco/{3_threads.md => 4-threads.md} (97%) rename docs/prog/pco/{4_verrous_semaphores.md => 5-verrous-semaphores.md} (98%) rename docs/prog/pco/{5_producteurs_consommateurs.md => 6-producteurs-consommateurs.md} (97%) rename docs/prog/pco/{6_moniteurs.md => 7-moniteurs.md} (98%) rename docs/prog/pco/{7_lecteurs_redacteurs.md => 8-lecteurs-redacteurs.md} (98%) rename docs/prog/pco/{summary.md => resume.md} (100%) rename docs/prog/{python-cheat-sheet.md => python-cheatsheet.md} (92%) diff --git a/docs/index.md b/docs/index.md index 6217cb9..3bcee9e 100644 --- a/docs/index.md +++ b/docs/index.md @@ -5,3 +5,15 @@ Midly interesting articles: * [Configuring `sshpass` with `gpg` & Zsh completion](softwares/sshpass.md) * [SUID, GUID & Sticky bits](os/linux/suid.md) * [Maven basics](softwares/maven.md) + +Code examples: + +* [JDBC: showing different ways of obtaining a database connection](prog/java/jdbc.md) +* [Log4j2: logging events in different files based on ThreadContext variables](prog/java/log4j2.md) + +Gist: + +* [libsodium: creating a password-derived key](https://gist.github.com/nyg/e366c27a70a77bf06581a0e6a8211cc9) +* [Swift: adding a JPEG comment marker to an image file](https://gist.github.com/nyg/bdeae8190a41b4b56bde8e13dd471ecc) +* [Swift & ImageIO: getting and setting an EXIF UserComment of a JPEG image](https://gist.github.com/nyg/c90f36abbd30f72c8b6681ef23db886b) +* macOS: getting boot time and uptime in [C](https://gist.github.com/nyg/dbdef21a1a0632c389d4d756d4fc1c0d) and in [Swift](https://gist.github.com/nyg/d81308a92fbf7e9c44c5f72db5ee2171) diff --git a/docs/prog/apollo-graphql.md b/docs/prog/apollo-graphql.md index 6b5ef0e..ba45eed 100644 --- a/docs/prog/apollo-graphql.md +++ b/docs/prog/apollo-graphql.md @@ -1,4 +1,6 @@ -# Apollo GraphQL +--- +title: Apollo GraphQL +--- * Website: [https://www.apollographql.com/](https://www.apollographql.com/) * Apollo Client docs: [https://www.apollographql.com/docs/react/](https://www.apollographql.com/docs/react/) diff --git a/docs/prog/res/java-io-streams.md b/docs/prog/java/io-streams.md similarity index 99% rename from docs/prog/res/java-io-streams.md rename to docs/prog/java/io-streams.md index 7c77b3d..b58e848 100644 --- a/docs/prog/res/java-io-streams.md +++ b/docs/prog/java/io-streams.md @@ -1,4 +1,6 @@ -# java-io-streams +--- +title: IO streams +--- Summary of the [Java I/O tutorial](https://docs.oracle.com/javase/tutorial/essential/io/index.html). diff --git a/docs/prog/java/jdbc.md b/docs/prog/java/jdbc.md new file mode 100644 index 0000000..a9c8e05 --- /dev/null +++ b/docs/prog/java/jdbc.md @@ -0,0 +1,128 @@ +--- +title: JDBC connection +--- + +*Source repository: [git.sr.ht/~nyg/example-java-jdbc](https://git.sr.ht/~nyg/example-java-jdbc)* + +There are different ways to obtain a connection to a database in a Java application. We will show some of them below, starting from the most low-level one, up to the most commonly used one. + +In this demo application, we obtain a connection to a MariaDB database. We use the `mariadb-java-client` dependency, which contains the MariaDB JDBC driver. + +## Java Service Provider Interface + +Simply put, the Java SPI is a JDK API for discovering and loading implementations of a given interface. The interface is the _service_ and the implementation is the _service provider_. + +In our case, the service is the `java.sql.Driver` interface and the provider is the `org.mariadb.jdbc.Driver ` class (which implements the Driver interface). + +In order for a provider to be discovered, its JAR file must include a file named `META-INF/services/` which contains the fully qualified classname of the provider class. For example, in the `mariadb-java-client.jar` we can find the file `META-INF/services/java.sql.Driver` which contains `org.mariadb.jdbc.Driver`. + +We use the the `ServiceLoader` class to find and instantiate the MariaDB driver. Once we have the driver instance, we can obtain a `java.sql.Connection` object which allows us to communicate with the underlying database. + +```java +ServiceLoader serviceLoader = ServiceLoader.load(java.sql.Driver.class); +serviceLoader.iterator().forEachRemaining(driver -> { + + String jdbcUrl = "jdbc:mariadb://localhost/mydb"; + Properties properties = new Properties(); + properties.setProperty("user", "admin"); + properties.setProperty("password", "admin1234"); + + Connection connection = driver.connect(jdbcUrl, properties); + // query db, e.g. connection.prepareStatement(…) +}); +``` + +### Before SPI + +The Java SPI was introduced in Java SE 6 and, since JDBC 4.0, the `java.sql.DriverManager` uses it to find and load available drivers (see below). Before that, the drivers had to be loaded manually as follows: + +```java +// load the driver class, which will call the driver's static initializer +Class.forName("org.mariadb.jdbc.Driver"); + +// obtain a driver instance either with Class.forName(…).newInstance() +// or by using the DriverManager class (see below) +``` + +The driver's static initializer registers the driver with the DriverManager, as explained in the JDBC specification: + +> JDBC drivers must implement the Driver interface, and the implementation must contain a static initializer that will be called when the driver is loaded. This initializer registers a new instance of itself with the DriverManager. + +Resources: + +* Javadoc: [ServiceLoader](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/ServiceLoader.html), [Driver](https://docs.oracle.com/en/java/javase/17/docs/api/java.sql/java/sql/Driver.html) +* [marcobehler.com: JDBC - a short guide](https://www.marcobehler.com/guides/jdbc) + +## DriverManager + +As explained above, the DriverManager uses SPI to discover and load drivers available on the classpath. We can query the DriverManager for registered drivers using the `getDrivers()` method, or directly call its `getConnection()` method. + +```java +Connection connection = DriverManager.getConnection(jdbcUrl, properties); +// query db, e.g. connection.prepareStatement(…) +``` + +The `getConnection()` method will loop through the registered drivers and return a connection from the first one which accepts the provided JDBC URL. + +Resources: + +* Javadoc: [DriverManager](https://docs.oracle.com/en/java/javase/17/docs/api/java.sql/java/sql/DriverManager.html) + +## DataSource + +As mentioned in the Javadoc for the DriverManager class, “the use of a `DataSource` object is the preferred means of connecting to a data source”. There are, of course, multiple ways to obtain a DataSource object. + +### Driver DataSource + +If the driver provides a DataSource implementation, it could be used directly: + +```java +MariaDbDataSource dataSource = new MariaDbDataSource(JDBC_URL); +dataSource.setUser("admin"); +dataSource.setPassword("admin1234"); + +Connection connection = dataSource.getConnection(); +// query db, e.g. connection.prepareStatement(…) +``` + +However, in a real-world app, using a connection pool is preferred for performance reasons. Obtaining a database connection is an expensive operation. + +### JDBC Connection Pool + +There are multiple JDBC connection pools out there, one of them is the Tomcat JDBC Connection Pool (dependency `tomcat-jdbc`). It can be used in the following way: + +```java +PoolConfiguration poolConfig = new PoolProperties(); +poolConfig.setUrl(jdbcUrl); +poolConfig.setDriverClassName("org.mariadb.jdbc.Driver"); +poolConfig.setUsername("admin"); +poolConfig.setPassword("admin1234"); +// much more pool configuration properties available + +DataSource = new org.apache.tomcat.jdbc.pool.DataSource(poolConfig); +``` + +Tomcat JDBC Connection Pool can be used in a standalone application, it does not require the Tomcat web server. + +### Java Naming and Directory Interface + +In a real-world application, if the application runs inside an application server (e.g. Tomcat), the DataSource is defined in the server configuration and is exposed in a JNDI context. This allows the application to retrieve the DataSource using the JNDI resource name: + +```java +// either "manually" +DataSource jndiDataSource = (DataSource) new InitialContext().lookup("java:comp/env/jdbc/my-datasource"); + +// or using the Jakarta EE @Resource annotation +@Resource(lookup = "java:comp/env/jdbc/my-datasource") +private DataSource jndiDataSource; +``` + +If JPA is also used in the application (e.g. with Spring and/or Hibernate), the DataSource instance will be given to an EntityManagerFactory (JPA) or SessionFactory (Hibernate). In this case, the connection is obtained behind the scene, when it is needed. + +Resources: + +* Stackoverflow + * [Why do we use a DataSource instead of a DriverManager?](https://stackoverflow.com/a/73148345) + * [Why use JNDI for data sources?](https://stackoverflow.com/a/7760768) +* [https://tomcat.apache.org/tomcat-10.1-doc/jdbc-pool.html](https://tomcat.apache.org/tomcat-10.1-doc/jdbc-pool.html) +* Javadoc: [@Resource](https://jakarta.ee/specifications/platform/10/apidocs/jakarta/annotation/resource) diff --git a/docs/prog/java/log4j2.md b/docs/prog/java/log4j2.md new file mode 100644 index 0000000..e65491d --- /dev/null +++ b/docs/prog/java/log4j2.md @@ -0,0 +1,95 @@ +--- +title: Log4j2 ThreadContext +--- + +*Source repository: [git.sr.ht/~nyg/example-java-log4j](https://git.sr.ht/~nyg/example-java-log4j)* + +This proof-of-concept shows how to use a Log4j `Routing` appender to route log events of a service to different files, based on who initially called the service methods, i.e. a JMX call or a Spring scheduler in our case. + +The service `Service` has two methods: `doOne` and `doTwo`, which represent two distinct business actions in an application. Each method is called by a scheduler every 15 seconds in `SchedulerOne` and `SchedulerTwo`, respectively. Each method can also be executed via a JMX call through the `OperationOne` and `OperationTwo` managed resources (e.g. by using VisualVM). + +As the methods could be executed asynchronously and/or may take considerable time to execute (in a real application), the logs produced by each method would overlap each other if logged into the same file. This makes it hard to distinguish the execution flow of each method invocation and debug any problems that might arise in a production environment. + +## ThreadContext + +By using the log4j `ThreadContext`, we can associate multiple key-value pairs to a thread in order to better identify each method invocation. For example: + +```java +@ManagedOperation +public void executeOne(String data) { + ThreadContext.put("caller", "jmx"); + ThreadContext.put("operation", "one"); + ThreadContext.put("uuid", UUID.randomUUID().toString().split("-")[0]); + + log.info("Starting Operation One with: {}", data); + service.doOne(data); + + ThreadContext.clearMap(); +} +``` + +```java +@Scheduled(fixedRate = 15_000) +public void schedulerOne() { + ThreadContext.put("caller", "sched"); + ThreadContext.put("operation", "one"); + ThreadContext.put("uuid", UUID.randomUUID().toString().split("-")[0]); + + log.info("Executing Scheduler One"); + service.doOne("no-data"); + + ThreadContext.clearMap(); +} +``` + +## Dynamic filenames + +The key-value pairs can be used in the used in the log4j configuration file in the following way: + +```xml + + + + + + + +``` + +Using this configuration, when the `doOne` method is invoked via JMX, the logs are written to a file named `example-log4j-jmx-one.log`. When it is invoked by the scheduler, the filename is `example-log4j-sched-one.log`. The same is done for the `doTwo` method. + +See below for the value of the `extendedPattern` property. + +## Log format + +The key-value pairs can also be used in the `PatternLayout`, so as to be able to distinguish each log event easily, even if they were written to a single log file: + +```xml + + %d{ISO8601} [%c{1.1.1.2.}] [%t] [%X{caller}-%X{operation}-%X{uuid}] [%p] %m%n + +``` + +Example of `example-log4j-jmx-one.log`: + +``` +2023-03-19T20:15:14,605 [e.s.n.ex.lo.ap.jm.OperationOne] [RMI TCP Connection(4)-192.168.1.213] [jmx-one-d5458d68] [INFO] Starting Operation One with: some data +2023-03-19T20:15:14,618 [e.s.n.ex.lo.ap.co.Service] [RMI TCP Connection(4)-192.168.1.213] [jmx-one-d5458d68] [INFO] Executed One with: some data +2023-03-19T20:15:18,618 [e.s.n.ex.lo.ap.co.Service] [Thread-26] [jmx-one-d5458d68] [INFO] This is logged from a separate thread but inherits from the ThreadContext map +2023-03-19T20:15:53,934 [e.s.n.ex.lo.ap.jm.OperationOne] [RMI TCP Connection(4)-192.168.1.213] [jmx-one-e6a8d238] [INFO] Starting Operation One with: some other data +2023-03-19T20:15:53,940 [e.s.n.ex.lo.ap.co.Service] [RMI TCP Connection(4)-192.168.1.213] [jmx-one-e6a8d238] [INFO] Executed One with: some other data +2023-03-19T20:15:57,941 [e.s.n.ex.lo.ap.co.Service] [Thread-31] [jmx-one-e6a8d238] [INFO] This is logged from a separate thread but inherits from the ThreadContext map +``` + +Example of `example-log4j-sched-two.log`: + +``` +2023-03-19T20:12:57,851 [e.s.n.ex.lo.ap.sc.SchedulerTwo] [sched-task-2] [sched-two-09f481d1] [INFO] Executing Scheduler Two +2023-03-19T20:12:57,852 [e.s.n.ex.lo.ap.co.Service] [sched-task-2] [sched-two-09f481d1] [INFO] Executed Two with: no-data +2023-03-19T20:13:01,858 [e.s.n.ex.lo.ap.co.Service] [Thread-6] [sched-two-09f481d1] [INFO] This is logged from a separate thread but inherits from the ThreadContext map +2023-03-19T20:13:12,850 [e.s.n.ex.lo.ap.sc.SchedulerTwo] [sched-task-7] [sched-two-bcbd80d6] [INFO] Executing Scheduler Two +2023-03-19T20:13:12,852 [e.s.n.ex.lo.ap.co.Service] [sched-task-7] [sched-two-bcbd80d6] [INFO] Executed Two with: no-data +2023-03-19T20:13:16,856 [e.s.n.ex.lo.ap.co.Service] [Thread-8] [sched-two-bcbd80d6] [INFO] This is logged from a separate thread but inherits from the ThreadContext map +``` + +As we can see, each log event contains a string of the following format: `[--]`, e.g. `sched-two-00be3fa0`. diff --git a/docs/prog/pco/1_exclusion_mutuelle.md b/docs/prog/pco/2-exclusion-mutuelle.md similarity index 97% rename from docs/prog/pco/1_exclusion_mutuelle.md rename to docs/prog/pco/2-exclusion-mutuelle.md index 54bd840..7ad3525 100644 --- a/docs/prog/pco/1_exclusion_mutuelle.md +++ b/docs/prog/pco/2-exclusion-mutuelle.md @@ -1,4 +1,6 @@ -# 1 – Attente active +--- +title: Exclusion mutuelle +--- * Une **ressource critique** est une ressource qui ne doit être accédée par une seule tâche à l'instant t. * La portion de code accédant à une ressource critique est appelée **section critique**. @@ -52,4 +54,3 @@ void T0::run() { } } ``` - diff --git a/docs/prog/pco/3_threads.md b/docs/prog/pco/4-threads.md similarity index 97% rename from docs/prog/pco/3_threads.md rename to docs/prog/pco/4-threads.md index 98732e1..4f072d0 100644 --- a/docs/prog/pco/3_threads.md +++ b/docs/prog/pco/4-threads.md @@ -1,4 +1,6 @@ -# 3 – Threads +--- +title: Threads +--- * **Green thread** : thread ordonnancé par une machine virtuelle ou une librairie qui émule l'ordonnanceur de l'OS. * **Native thread** : thread natif géré par l'ordonnanceur de l'OS. @@ -35,4 +37,3 @@ QThread::isInterruptionRequested() // depuis le thread concerné QThread::currentThreadId() // retourne l'id the thread courrant QThread::yieldCurrentThread() // le thread appelant est préempté ``` - diff --git a/docs/prog/pco/4_verrous_semaphores.md b/docs/prog/pco/5-verrous-semaphores.md similarity index 98% rename from docs/prog/pco/4_verrous_semaphores.md rename to docs/prog/pco/5-verrous-semaphores.md index a264651..642389e 100644 --- a/docs/prog/pco/4_verrous_semaphores.md +++ b/docs/prog/pco/5-verrous-semaphores.md @@ -1,4 +1,6 @@ -# 4 – Verrous et Sémaphores +--- +title: Verrous et Sémaphores +--- ## Verrous @@ -69,4 +71,3 @@ QSemaphore::release(int n = 1) QSemaphore::tryAcquire(int n = 1) QSemaphore::available() ``` - diff --git a/docs/prog/pco/5_producteurs_consommateurs.md b/docs/prog/pco/6-producteurs-consommateurs.md similarity index 97% rename from docs/prog/pco/5_producteurs_consommateurs.md rename to docs/prog/pco/6-producteurs-consommateurs.md index 13cb9ec..cfb0859 100644 --- a/docs/prog/pco/5_producteurs_consommateurs.md +++ b/docs/prog/pco/6-producteurs-consommateurs.md @@ -1,4 +1,6 @@ -# 5 – Producteurs-consommateurs +--- +title: Producteurs-consommateurs +--- * Un thread **producteur** et un thread **consommateur** communiquent un utilisant un tampon. * Seul le producteur écrit et seul le consommateur lit dans le tampon. @@ -68,4 +70,3 @@ T get(void) { return item; } ``` - diff --git a/docs/prog/pco/6_moniteurs.md b/docs/prog/pco/7-moniteurs.md similarity index 98% rename from docs/prog/pco/6_moniteurs.md rename to docs/prog/pco/7-moniteurs.md index 86d928b813657de6a240e3c213b93b0646873b68..433b9ad87fc9ea18bd1cd38bc9c9969a6727922c 100644 GIT binary patch delta 33 ncmbOwxIj=sS67#-B(o$Z)k?uPKQFT+wX~?13n;izu!92ts_6 diff --git a/docs/prog/res/udp.md b/docs/prog/res/udp.md index d088d72..ccc51d5 100644 --- a/docs/prog/res/udp.md +++ b/docs/prog/res/udp.md @@ -1,4 +1,6 @@ -# UDP – User Datagram Protocol +--- +title: User Datagram Protocol +--- ## C API @@ -67,4 +69,3 @@ socket.joinGroup(multicastGroup); * Service provider publish datagram with contact details in multicast group every 5 seconds. A client joins the multicast group and receives the datagrams. * The service provider joins a multicast group and the laptops send service request datagrams. Printers then contact the laptops individually with contact details. - diff --git a/docs/prog/resume-tweb.md b/docs/prog/resume-tweb.md index c16794d..376eede 100644 --- a/docs/prog/resume-tweb.md +++ b/docs/prog/resume-tweb.md @@ -1,5 +1,5 @@ --- -title: Résumé — Technologie Web +title: Résumé --- ## URL parts diff --git a/docs/prog/sagemath.md b/docs/prog/sagemath.md index a2598a7..8e8528c 100644 --- a/docs/prog/sagemath.md +++ b/docs/prog/sagemath.md @@ -1,6 +1,6 @@ -# SageMath - -## Issues +--- +title: SageMath +--- * Unable to run `sage -pip install` due to issues with TLS. ```sh diff --git a/docs/softwares/maven.md b/docs/softwares/maven.md index 6aaade1..fd4fa82 100644 --- a/docs/softwares/maven.md +++ b/docs/softwares/maven.md @@ -2,7 +2,7 @@ title: Maven --- -*Notes from Maven: The Definitive Guide* +Notes from the book *Maven: The Definitive Guide*. Additional resource: [git.sr.ht/~nyg/maven-basics](https://git.sr.ht/~nyg/maven-basics). Maven is known as a "build tool" but it can do more than just building and packaging a project: diff --git a/mkdocs.yml b/mkdocs.yml index ebd1ad4..2acd270 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -65,23 +65,26 @@ nav: - Welcome: index.md - Tags: tags.md - Programmation: - - Apollo GraphQL: prog/apollo-graphql.md - - Python: prog/python-cheat-sheet.md - - SageMath: prog/sagemath.md + - prog/apollo-graphql.md + - prog/python-cheatsheet.md + - prog/sagemath.md + - Java: + - prog/java/jdbc.md + - prog/java/log4j2.md + - prog/java/io-streams.md - Concurrency: - - Résumé: prog/pco/summary.md - - Exclusion mutuelle: prog/pco/1_exclusion_mutuelle.md - - Threads: prog/pco/3_threads.md - - Verrous & sémaphores: prog/pco/4_verrous_semaphores.md - - Producteurs & consommateurs: prog/pco/5_producteurs_consommateurs.md - - Moniteurs: prog/pco/6_moniteurs.md - - Lecteurs & Rédacteurs: prog/pco/7_lecteurs_redacteurs.md + - prog/pco/resume.md + - prog/pco/2-exclusion-mutuelle.md + - prog/pco/4-threads.md + - prog/pco/5-verrous-semaphores.md + - prog/pco/6-producteurs-consommateurs.md + - prog/pco/7-moniteurs.md + - prog/pco/8-lecteurs-redacteurs.md - Network: - - Java IO Streams: prog/res/java-io-streams.md - - SMTP: prog/res/smtp.md - - UDP: prog/res/udp.md + - prog/res/smtp.md + - prog/res/udp.md - Web: - - Résumé: prog/resume-tweb.md + - prog/resume-tweb.md - Softwares: - Git: - softwares/git.md -- 2.45.2