~nyg/notes

d2ed8585efa444eb8c2814f7d024154e446eb244 — nyg 9 months ago 15b9bbc master
Add jdbc and log4j code examples
19 files changed, 287 insertions(+), 34 deletions(-)

M docs/index.md
M docs/prog/apollo-graphql.md
R docs/prog/{res/java-io-streams.md => java/io-streams.md}
A docs/prog/java/jdbc.md
A docs/prog/java/log4j2.md
R docs/prog/pco/{1_exclusion_mutuelle.md => 2-exclusion-mutuelle.md}
R docs/prog/pco/{3_threads.md => 4-threads.md}
R docs/prog/pco/{4_verrous_semaphores.md => 5-verrous-semaphores.md}
R docs/prog/pco/{5_producteurs_consommateurs.md => 6-producteurs-consommateurs.md}
R docs/prog/pco/{6_moniteurs.md => 7-moniteurs.md}
R docs/prog/pco/{7_lecteurs_redacteurs.md => 8-lecteurs-redacteurs.md}
R docs/prog/pco/{summary.md => resume.md}
R docs/prog/{python-cheat-sheet.md => python-cheatsheet.md}
M docs/prog/res/smtp.md
M docs/prog/res/udp.md
M docs/prog/resume-tweb.md
M docs/prog/sagemath.md
M docs/softwares/maven.md
M mkdocs.yml
M docs/index.md => docs/index.md +12 -0
@@ 5,3 5,15 @@ Midly interesting articles:
* [Configuring `sshpass` with `gpg` & Zsh completion](softwares/sshpass.md)
* [SUID, GUID & Sticky bits](os/linux/suid.md)
* [Maven basics](softwares/maven.md)

Code examples:

* [JDBC: showing different ways of obtaining a database connection](prog/java/jdbc.md)
* [Log4j2: logging events in different files based on ThreadContext variables](prog/java/log4j2.md)

Gist:

* [libsodium: creating a password-derived key](https://gist.github.com/nyg/e366c27a70a77bf06581a0e6a8211cc9)
* [Swift: adding a JPEG comment marker to an image file](https://gist.github.com/nyg/bdeae8190a41b4b56bde8e13dd471ecc)
* [Swift & ImageIO: getting and setting an EXIF UserComment of a JPEG image](https://gist.github.com/nyg/c90f36abbd30f72c8b6681ef23db886b)
* macOS: getting boot time and uptime in [C](https://gist.github.com/nyg/dbdef21a1a0632c389d4d756d4fc1c0d) and in [Swift](https://gist.github.com/nyg/d81308a92fbf7e9c44c5f72db5ee2171)

M docs/prog/apollo-graphql.md => docs/prog/apollo-graphql.md +3 -1
@@ 1,4 1,6 @@
# Apollo GraphQL
---
title: Apollo GraphQL
---

* Website: [https://www.apollographql.com/](https://www.apollographql.com/)
* Apollo Client docs: [https://www.apollographql.com/docs/react/](https://www.apollographql.com/docs/react/)

R docs/prog/res/java-io-streams.md => docs/prog/java/io-streams.md +3 -1
@@ 1,4 1,6 @@
# java-io-streams
---
title: IO streams
---

Summary of the [Java I/O tutorial](https://docs.oracle.com/javase/tutorial/essential/io/index.html).


A docs/prog/java/jdbc.md => docs/prog/java/jdbc.md +128 -0
@@ 0,0 1,128 @@
---
title: JDBC connection
---

*Source repository: [git.sr.ht/~nyg/example-java-jdbc](https://git.sr.ht/~nyg/example-java-jdbc)*

There are different ways to obtain a connection to a database in a Java application. We will show some of them below, starting from the most low-level one, up to the most commonly used one.

In this demo application, we obtain a connection to a MariaDB database. We use the `mariadb-java-client` dependency, which contains the MariaDB JDBC driver.

## Java Service Provider Interface

Simply put, the Java SPI is a JDK API for discovering and loading implementations of a given interface. The interface is the _service_ and the implementation is the _service provider_.

In our case, the service is the `java.sql.Driver` interface and the provider is the `org.mariadb.jdbc.Driver ` class (which implements the Driver interface).

In order for a provider to be discovered, its JAR file must include a file named `META-INF/services/<fully-qualified-interface-name>` which contains the fully qualified classname of the provider class. For example, in the `mariadb-java-client.jar` we can find the file `META-INF/services/java.sql.Driver` which contains `org.mariadb.jdbc.Driver`.

We use the the `ServiceLoader` class to find and instantiate the MariaDB driver. Once we have the driver instance, we can obtain a `java.sql.Connection` object which allows us to communicate with the underlying database.

```java
ServiceLoader<Driver> serviceLoader = ServiceLoader.load(java.sql.Driver.class);
serviceLoader.iterator().forEachRemaining(driver -> {

    String jdbcUrl = "jdbc:mariadb://localhost/mydb";
    Properties properties = new Properties();
    properties.setProperty("user", "admin");
    properties.setProperty("password", "admin1234");

    Connection connection = driver.connect(jdbcUrl, properties);
    // query db, e.g. connection.prepareStatement(…)
});
```

### Before SPI

The Java SPI was introduced in Java SE 6 and, since JDBC 4.0, the `java.sql.DriverManager` uses it to find and load available drivers (see below). Before that, the drivers had to be loaded manually as follows:

```java
// load the driver class, which will call the driver's static initializer
Class.forName("org.mariadb.jdbc.Driver");

// obtain a driver instance either with Class.forName(…).newInstance()
// or by using the DriverManager class (see below)
```

The driver's static initializer registers the driver with the DriverManager, as explained in the JDBC specification:

>  JDBC drivers must implement the Driver interface, and the implementation must contain a static initializer that will be called when the driver is loaded. This initializer registers a new instance of itself with the DriverManager.

Resources:

* Javadoc: [ServiceLoader](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/ServiceLoader.html), [Driver](https://docs.oracle.com/en/java/javase/17/docs/api/java.sql/java/sql/Driver.html)
* [marcobehler.com: JDBC - a short guide](https://www.marcobehler.com/guides/jdbc)

## DriverManager

As explained above, the DriverManager uses SPI to discover and load drivers available on the classpath. We can query the DriverManager for registered drivers using the `getDrivers()` method, or directly call its `getConnection()` method.

```java
Connection connection = DriverManager.getConnection(jdbcUrl, properties);
// query db, e.g. connection.prepareStatement(…)
```

The `getConnection()` method will loop through the registered drivers and return a connection from the first one which accepts the provided JDBC URL.

Resources:

* Javadoc: [DriverManager](https://docs.oracle.com/en/java/javase/17/docs/api/java.sql/java/sql/DriverManager.html)

## DataSource

As mentioned in the Javadoc for the DriverManager class, “the use of a `DataSource` object is the preferred means of connecting to a data source”. There are, of course, multiple ways to obtain a DataSource object.

### Driver DataSource

If the driver provides a DataSource implementation, it could be used directly:

```java
MariaDbDataSource dataSource = new MariaDbDataSource(JDBC_URL);
dataSource.setUser("admin");
dataSource.setPassword("admin1234");

Connection connection = dataSource.getConnection();
// query db, e.g. connection.prepareStatement(…)
```

However, in a real-world app, using a connection pool is preferred for performance reasons. Obtaining a database connection is an expensive operation.

### JDBC Connection Pool

There are multiple JDBC connection pools out there, one of them is the Tomcat JDBC Connection Pool (dependency `tomcat-jdbc`). It can be used in the following way:

```java
PoolConfiguration poolConfig = new PoolProperties();
poolConfig.setUrl(jdbcUrl);
poolConfig.setDriverClassName("org.mariadb.jdbc.Driver");
poolConfig.setUsername("admin");
poolConfig.setPassword("admin1234");
// much more pool configuration properties available

DataSource = new org.apache.tomcat.jdbc.pool.DataSource(poolConfig);
```

Tomcat JDBC Connection Pool can be used in a standalone application, it does not require the Tomcat web server.

### Java Naming and Directory Interface

In a real-world application, if the application runs inside an application server (e.g. Tomcat), the DataSource is defined in the server configuration and is exposed in a JNDI context. This allows the application to retrieve the DataSource using the JNDI resource name:

```java
// either "manually"
DataSource jndiDataSource = (DataSource) new InitialContext().lookup("java:comp/env/jdbc/my-datasource");

// or using the Jakarta EE @Resource annotation
@Resource(lookup = "java:comp/env/jdbc/my-datasource")
private DataSource jndiDataSource;
```

If JPA is also used in the application (e.g. with Spring and/or Hibernate), the DataSource instance will be given to an EntityManagerFactory (JPA) or SessionFactory (Hibernate). In this case, the connection is obtained behind the scene, when it is needed.

Resources:

* Stackoverflow
    * [Why do we use a DataSource instead of a DriverManager?](https://stackoverflow.com/a/73148345)
    * [Why use JNDI for data sources?](https://stackoverflow.com/a/7760768)
* [https://tomcat.apache.org/tomcat-10.1-doc/jdbc-pool.html](https://tomcat.apache.org/tomcat-10.1-doc/jdbc-pool.html)
* Javadoc: [@Resource](https://jakarta.ee/specifications/platform/10/apidocs/jakarta/annotation/resource)

A docs/prog/java/log4j2.md => docs/prog/java/log4j2.md +95 -0
@@ 0,0 1,95 @@
---
title: Log4j2 ThreadContext
---

*Source repository: [git.sr.ht/~nyg/example-java-log4j](https://git.sr.ht/~nyg/example-java-log4j)*

This proof-of-concept shows how to use a Log4j `Routing` appender to route log events of a service to different files, based on who initially called the service methods, i.e. a JMX call or a Spring scheduler in our case.

The service `Service` has two methods: `doOne` and `doTwo`, which represent two distinct business actions in an application. Each method is called by a scheduler every 15 seconds in `SchedulerOne` and `SchedulerTwo`, respectively. Each method can also be executed via a JMX call through the `OperationOne` and `OperationTwo` managed resources (e.g. by using VisualVM).

As the methods could be executed asynchronously and/or may take considerable time to execute (in a real application), the logs produced by each method would overlap each other if logged into the same file. This makes it hard to distinguish the execution flow of each method invocation and debug any problems that might arise in a production environment.

## ThreadContext

By using the log4j `ThreadContext`, we can associate multiple key-value pairs to a thread in order to better identify each method invocation. For example:

```java
@ManagedOperation
public void executeOne(String data) {
    ThreadContext.put("caller", "jmx");
    ThreadContext.put("operation", "one");
    ThreadContext.put("uuid", UUID.randomUUID().toString().split("-")[0]);

    log.info("Starting Operation One with: {}", data);
    service.doOne(data);

    ThreadContext.clearMap();
}
```

```java
@Scheduled(fixedRate = 15_000)
public void schedulerOne() {
    ThreadContext.put("caller", "sched");
    ThreadContext.put("operation", "one");
    ThreadContext.put("uuid", UUID.randomUUID().toString().split("-")[0]);

    log.info("Executing Scheduler One");
    service.doOne("no-data");

    ThreadContext.clearMap();
}
```

## Dynamic filenames

The key-value pairs can be used in the used in the log4j configuration file in the following way:

```xml
<Routes pattern="$${ctx:caller}-$${ctx:operation}">
    <Route>
        <File name="File-${ctx:caller}-${ctx:operation}" fileName="logs/example-log4j-${ctx:caller}-${ctx:operation}.log">
            <PatternLayout pattern="${extendedPattern}"/>
        </File>
    </Route>
</Routes>
```

Using this configuration, when the `doOne` method is invoked via JMX, the logs are written to a file named `example-log4j-jmx-one.log`. When it is invoked by the scheduler, the filename is `example-log4j-sched-one.log`. The same is done for the `doTwo` method.

See below for the value of the `extendedPattern` property.

## Log format

The key-value pairs can also be used in the `PatternLayout`, so as to be able to distinguish each log event easily, even if they were written to a single log file:

```xml
<Property name="extendedPattern">
    %d{ISO8601} [%c{1.1.1.2.}] [%t] [%X{caller}-%X{operation}-%X{uuid}] [%p] %m%n
</Property>
```

Example of `example-log4j-jmx-one.log`:

```
2023-03-19T20:15:14,605 [e.s.n.ex.lo.ap.jm.OperationOne] [RMI TCP Connection(4)-192.168.1.213] [jmx-one-d5458d68] [INFO] Starting Operation One with: some data
2023-03-19T20:15:14,618 [e.s.n.ex.lo.ap.co.Service] [RMI TCP Connection(4)-192.168.1.213] [jmx-one-d5458d68] [INFO] Executed One with: some data
2023-03-19T20:15:18,618 [e.s.n.ex.lo.ap.co.Service] [Thread-26] [jmx-one-d5458d68] [INFO] This is logged from a separate thread but inherits from the ThreadContext map
2023-03-19T20:15:53,934 [e.s.n.ex.lo.ap.jm.OperationOne] [RMI TCP Connection(4)-192.168.1.213] [jmx-one-e6a8d238] [INFO] Starting Operation One with: some other data
2023-03-19T20:15:53,940 [e.s.n.ex.lo.ap.co.Service] [RMI TCP Connection(4)-192.168.1.213] [jmx-one-e6a8d238] [INFO] Executed One with: some other data
2023-03-19T20:15:57,941 [e.s.n.ex.lo.ap.co.Service] [Thread-31] [jmx-one-e6a8d238] [INFO] This is logged from a separate thread but inherits from the ThreadContext map
```

Example of `example-log4j-sched-two.log`:

```
2023-03-19T20:12:57,851 [e.s.n.ex.lo.ap.sc.SchedulerTwo] [sched-task-2] [sched-two-09f481d1] [INFO] Executing Scheduler Two
2023-03-19T20:12:57,852 [e.s.n.ex.lo.ap.co.Service] [sched-task-2] [sched-two-09f481d1] [INFO] Executed Two with: no-data
2023-03-19T20:13:01,858 [e.s.n.ex.lo.ap.co.Service] [Thread-6] [sched-two-09f481d1] [INFO] This is logged from a separate thread but inherits from the ThreadContext map
2023-03-19T20:13:12,850 [e.s.n.ex.lo.ap.sc.SchedulerTwo] [sched-task-7] [sched-two-bcbd80d6] [INFO] Executing Scheduler Two
2023-03-19T20:13:12,852 [e.s.n.ex.lo.ap.co.Service] [sched-task-7] [sched-two-bcbd80d6] [INFO] Executed Two with: no-data
2023-03-19T20:13:16,856 [e.s.n.ex.lo.ap.co.Service] [Thread-8] [sched-two-bcbd80d6] [INFO] This is logged from a separate thread but inherits from the ThreadContext map
```

As we can see, each log event contains a string of the following format: `[<caller>-<operation-name>-<operation-uuid>]`, e.g. `sched-two-00be3fa0`.

R docs/prog/pco/1_exclusion_mutuelle.md => docs/prog/pco/2-exclusion-mutuelle.md +3 -2
@@ 1,4 1,6 @@
# 1 – Attente active
---
title: Exclusion mutuelle
---

* Une **ressource critique** est une ressource qui ne doit être accédée par une seule tâche à l'instant t.
* La portion de code accédant à une ressource critique est appelée **section critique**.


@@ 52,4 54,3 @@ void T0::run() {
    }
}
```


R docs/prog/pco/3_threads.md => docs/prog/pco/4-threads.md +3 -2
@@ 1,4 1,6 @@
# 3 – Threads
---
title: Threads
---

* **Green thread** : thread ordonnancé par une machine virtuelle ou une librairie qui émule l'ordonnanceur de l'OS.
* **Native thread** : thread natif géré par l'ordonnanceur de l'OS.


@@ 35,4 37,3 @@ QThread::isInterruptionRequested() // depuis le thread concerné
QThread::currentThreadId()    // retourne l'id the thread courrant
QThread::yieldCurrentThread() // le thread appelant est préempté
```


R docs/prog/pco/4_verrous_semaphores.md => docs/prog/pco/5-verrous-semaphores.md +3 -2
@@ 1,4 1,6 @@
# 4 – Verrous et Sémaphores
---
title: Verrous et Sémaphores
---

## Verrous



@@ 69,4 71,3 @@ QSemaphore::release(int n = 1)
QSemaphore::tryAcquire(int n = 1)
QSemaphore::available()
```


R docs/prog/pco/5_producteurs_consommateurs.md => docs/prog/pco/6-producteurs-consommateurs.md +3 -2
@@ 1,4 1,6 @@
# 5 – Producteurs-consommateurs
---
title: Producteurs-consommateurs
---

* Un thread **producteur** et un thread **consommateur** communiquent un utilisant un tampon.
* Seul le producteur écrit et seul le consommateur lit dans le tampon.


@@ 68,4 70,3 @@ T get(void) {
    return item;
}
```


R docs/prog/pco/6_moniteurs.md => docs/prog/pco/7-moniteurs.md +0 -0
R docs/prog/pco/7_lecteurs_redacteurs.md => docs/prog/pco/8-lecteurs-redacteurs.md +3 -1
@@ 1,4 1,6 @@
# 7 – Lecteurs–rédacteurs
---
title: Lecteurs–rédacteurs
---

* Utilisé lorsque plusieurs threads doivent accéder à une ressource.
* Les lecteurs ne peuvent que lire et les rédacteurs qu'écrire.

R docs/prog/pco/summary.md => docs/prog/pco/resume.md +0 -0
R docs/prog/python-cheat-sheet.md => docs/prog/python-cheatsheet.md +3 -1
@@ 1,4 1,6 @@
## Python Cheat Sheet
---
title: Python cheatsheet
---

```py
# int bases

M docs/prog/res/smtp.md => docs/prog/res/smtp.md +3 -1
@@ 1,4 1,6 @@
# SMTP
---
title: Simple Mail Transfer Protocol
---

```
EHLO <user>

M docs/prog/res/udp.md => docs/prog/res/udp.md +3 -2
@@ 1,4 1,6 @@
# UDP – User Datagram Protocol
---
title: User Datagram Protocol
---

## C API



@@ 67,4 69,3 @@ socket.joinGroup(multicastGroup);

* Service provider publish datagram with contact details in multicast group every 5 seconds. A client joins the multicast group and receives the datagrams.
* The service provider joins a multicast group and the laptops send service request datagrams. Printers then contact the laptops individually with contact details.


M docs/prog/resume-tweb.md => docs/prog/resume-tweb.md +1 -1
@@ 1,5 1,5 @@
---
title: Résumé — Technologie Web
title: Résumé
---

## URL parts

M docs/prog/sagemath.md => docs/prog/sagemath.md +3 -3
@@ 1,6 1,6 @@
# SageMath

## Issues
---
title: SageMath
---

* Unable to run `sage -pip install` due to issues with TLS.
  ```sh

M docs/softwares/maven.md => docs/softwares/maven.md +1 -1
@@ 2,7 2,7 @@
title: Maven
---

*Notes from Maven: The Definitive Guide*
Notes from the book *Maven: The Definitive Guide*. Additional resource: [git.sr.ht/~nyg/maven-basics](https://git.sr.ht/~nyg/maven-basics).

Maven is known as a "build tool" but it can do more than just building and packaging a project:


M mkdocs.yml => mkdocs.yml +17 -14
@@ 65,23 65,26 @@ nav:
    - Welcome: index.md
    - Tags: tags.md
  - Programmation:
      - Apollo GraphQL: prog/apollo-graphql.md
      - Python: prog/python-cheat-sheet.md
      - SageMath: prog/sagemath.md
      - prog/apollo-graphql.md
      - prog/python-cheatsheet.md
      - prog/sagemath.md
      - Java:
          - prog/java/jdbc.md
          - prog/java/log4j2.md
          - prog/java/io-streams.md
      - Concurrency:
          - Résumé: prog/pco/summary.md
          - Exclusion mutuelle: prog/pco/1_exclusion_mutuelle.md
          - Threads: prog/pco/3_threads.md
          - Verrous & sémaphores: prog/pco/4_verrous_semaphores.md
          - Producteurs & consommateurs: prog/pco/5_producteurs_consommateurs.md
          - Moniteurs: prog/pco/6_moniteurs.md
          - Lecteurs & Rédacteurs: prog/pco/7_lecteurs_redacteurs.md
          - prog/pco/resume.md
          - prog/pco/2-exclusion-mutuelle.md
          - prog/pco/4-threads.md
          - prog/pco/5-verrous-semaphores.md
          - prog/pco/6-producteurs-consommateurs.md
          - prog/pco/7-moniteurs.md
          - prog/pco/8-lecteurs-redacteurs.md
      - Network:
          - Java IO Streams: prog/res/java-io-streams.md
          - SMTP: prog/res/smtp.md
          - UDP: prog/res/udp.md
          - prog/res/smtp.md
          - prog/res/udp.md
      - Web:
          - Résumé: prog/resume-tweb.md
          - prog/resume-tweb.md
  - Softwares:
      - Git:
          - softwares/git.md