~cg/uls2sqlite

369e92af38287d433c342cd92d7f98f0625d667d — Chris Goff 3 months ago 5d2a36d
Fixed formatting.
1 files changed, 18 insertions(+), 8 deletions(-)

M README.md
M README.md => README.md +18 -8
@@ 15,7 15,7 @@ ULS2SQLite has been tested on [Fedora Linux 37](https://fedoraproject.org/), but

## Installation  

```
```bash
git clone https://git.sr.ht/~cg/ULS2SQLite
cd ULS2SQLite/
pip install -r requirements.txt


@@ 27,12 27,14 @@ This tool will process `.DAT` files from a daily or weekly download from the FCC

Command-line arguments to specify input and output files. Use the `-i` or `--import_file` flag to specify the ULS file to be imported, and the `-o` or `--output_file` flag to specify the SQLite database filename to export to. If no SQLite database exists, it will be created.

Example using ULS `.DAT` file as input:
`python uls2sqlite.py -i HD.dat -o output.sqlite --encoding=utf-8`
Example using ULS `.DAT` file as input:  
```bash
python uls2sqlite.py -i HD.dat -o output.sqlite --encoding=utf-8
```

Note search export files are also pipe-delimited, however they are a single file containing many ULS records. There is a helper tool included, `uls_splitter.py`, which can be run against these files to split them into separate pipe-delimited files.

Example using a single ULS file as input:
Example using a single ULS file as input:  

```bash
cd tools


@@ 46,7 48,9 @@ This will split the `UL20236291542491.txt` file into separate pipe-delimited fil
There is also a helper file, `process_zip_files.sh`, which can be used to process ZIP files downloaded from the FCC website located in the `archives` directory. This will process all the files for you in an automated manner.

Example using ZIP files downloaded to the `archives` directory:
`bash process_zip_files.sh`
```bash
bash process_zip_files.sh
```

### Helper Tools



@@ 55,7 59,10 @@ There are a number of helper tools included to assist with the conversion proces
#### process_zip_files.sh
Processes batches of ZIP files downloaded from the FCC website and converts them into SQLite databases.  

Usage: `bash process_zip_files.sh`
Usage:  
```bash
bash process_zip_files.sh
```

Note: If [GNU Parallel](https://www.gnu.org/software/parallel/) is installed on your system, the script will attempt to
process ZIP files in parallel for a massive speed boost. Be aware this can consume considerable system resources!


@@ 66,7 73,10 @@ Processes a single pipe-delimited file downloaded from the FCC website and conve
Processes a single pipe-delimited ULS database to a SQLite database. Run `uls_splitter.py` first to generate the necessary
pipe-delimited files.

Usage: `bash process_pipe_files.sh output.sqlite`
Usage:  
```bash 
bash process_pipe_files.sh output.sqlite
```

#### tools/download_weekly_dbs.sh
Downloads the weekly ULS databases from the URLs in `weekly_dbs_urls.txt`.


@@ 94,5 104,5 @@ ULS definitions SQL file:
ULS record types  
    https://www.fcc.gov/sites/default/files/pubacc_intro_11122014.pdf

ULS definitions:
ULS definitions:  
    https://www.fcc.gov/sites/default/files/uls_code_definitions_07182022.txt 
\ No newline at end of file