as.data.frame, ...
as.data.frame, ...
committed
Have Some Fun with ‘HTTP’ Headers
Methods are provided to retrieve and test ‘HTTP’ headers from a website. An ‘HTTP’ request and response header reference/explanatory data frame is also provided via the ‘hsecsecan’ project (https://github.com/riramar/hsecscan)
The following functions are implemented:
as.data.frame.insensitive
: Turn httr::headers()
objects into a
data frameassess_security_headers
: Assess “security” headers for a given URLexplain_headers
: Explain HTTP headers found from a URL requestexplore_app
: A Shiny App for Exploring HTTP Headershttp_headers
: HTTP Header Referenceinstall.packages("hdrs", repos = "https://cinc.rud.is/")
library(hdrs)
library(tibble) # for printing
# current version
packageVersion("hdrs")
## [1] '0.2.0'
httr::HEAD("https://rud.is/b") %>%
httr::headers() %>%
as.data.frame()
## # A tibble: 20 x 2
## name value
## <chr> <chr>
## 1 server nginx
## 2 date Wed, 06 Mar 2019 12:27:15 GMT
## 3 content-type text/html; charset=UTF-8
## 4 connection keep-alive
## 5 vary Accept-Encoding
## 6 set-cookie PHPSESSID=0f2uckd4t1tuf55hhecpk73i63; path=/
## 7 expires Thu, 07 Mar 2019 12:27:15 GMT
## 8 cache-control max-age=86400
## 9 pragma no-cache
## 10 link "<https://rud.is/b/wp-json/>; rel=\"https://api.w.org/\""
## 11 link <https://wp.me/23idr>; rel=shortlink
## 12 strict-transport-secu… max-age=31536000; includeSubDomains; preload
## 13 content-security-poli… default-src 'self' data: fonts.gstatic.com fonts.googleapis.com cdn.ampproject.org *.ampproje…
## 14 x-frame-options SAMEORIGIN
## 15 referrer-policy no-referrer-when-downgrade
## 16 feature-policy geolocation 'none';midi 'none';sync-xhr 'none';microphone 'none';camera 'none';magnetometer '…
## 17 x-powered-by <3
## 18 x-xss-protection 1; mode=block
## 19 x-content-type-options nosniff
## 20 content-encoding gzip
assess_security_headers("https://cran.r-project.org") %>%
dplyr::select(-url) %>%
gt::gt()
header |
value |
status_code |
message |
---|---|---|---|
access-control-allow-origin |
NA |
WARN |
Header not set |
content-security-policy |
NA |
WARN |
Header not set |
expect-ct |
NA |
WARN |
Header not set |
feature-policy |
NA |
WARN |
Header not set |
public-key-pins |
NA |
WARN |
Header not set |
referrer-policy |
NA |
WARN |
Header not set |
server |
Apache/2.4.10 (Debian) |
NOTE |
Server header found |
strict-transport-security |
NA |
WARN |
Header not set |
x-content-type-options |
NA |
WARN |
Header not set |
x-frame-options |
NA |
WARN |
Header not set |
x-permitted-cross-domain-policies |
NA |
WARN |
Header not set |
x-powered-by |
NA |
WARN |
Header not set |
x-xss-protection |
NA |
WARN |
Header not set |
Tsk, tsk…
assess_security_headers("https://rud.is/b") %>%
dplyr::select(-url) %>%
gt::gt()
header |
value |
status_code |
message |
---|---|---|---|
access-control-allow-origin |
NA |
WARN |
Header not set |
content-security-policy |
default-src ‘self’ data: fonts.gstatic.com fonts.googleapis.com cdn.ampproject.org *.ampproject.org *.w.org w.org wp.com sendpulse.com gravatar.com *.wp.com *.sendpulse.com *.gravatar.com wordpress.com *.wordpress.com http://prefetch.validatorsearch.verisignlabs.com/; script-src ‘self’ data: ‘unsafe-inline’ ‘unsafe-eval’ fonts.gstatic.com fonts.googleapis.com *.w.org w.org wp.com cdn.ampproject.org *.ampproject.org sendpulse.com gravatar.com *.wp.com *.sendpulse.com *.gravatar.com wordpress.com *.wordpress.com; style-src ‘self’ data: ‘unsafe-inline’ ‘unsafe-eval’ fonts.gstatic.com fonts.googleapis.com *.w.org w.org wp.com *.sendpulse.com sendpulse.com *.gravatar.com gravatar.com *.wp.com cdn.ampproject.org *.ampproject.org; |
OK |
NOTE: Policy present but not parse |
expect-ct |
NA |
WARN |
Header not set |
feature-policy |
geolocation ‘none’;midi ‘none’;sync-xhr ‘none’;microphone ‘none’;camera ‘none’;magnetometer ‘none’;gyroscope ‘none’;speaker ‘self’;fullscreen ‘self’;payment ‘none’; |
OK |
Value present but not verified |
public-key-pins |
NA |
WARN |
Header not set |
referrer-policy |
no-referrer-when-downgrade |
OK |
Acceptable setting found |
server |
nginx |
NOTE |
Server header found |
strict-transport-security |
max-age=31536000; includeSubDomains; preload |
OK |
Value present but not evaluated |
x-content-type-options |
nosniff |
OK |
Acceptable setting found |
x-frame-options |
SAMEORIGIN |
OK |
Acceptable setting found |
x-permitted-cross-domain-policies |
NA |
WARN |
Header not set |
x-powered-by |
<3 |
NOTE |
X-Powered-By header present |
x-xss-protection |
1; mode=block |
WARN |
Required value (‘nosniff’) not present |
Looks like I gots some ’splainin to do as well.
hdrs::explain_headers("https://community.rstudio.com/") %>%
dplyr::select(header,value, enable, security_reference, recommendations) %>%
gt::gt()
header |
value |
enable |
security_reference |
recommendations |
---|---|---|---|---|
cache-control |
no-cache, no-store |
TRUE |
Do not store unnecessarily sensitive information in the cache. |
|
connection |
keep-alive |
FALSE |
||
content-encoding |
gzip |
TRUE |
Another suggested approach is to disable HTTP compression whenever the referrer header indicates a cross-site request, or when the header is not present. This approach allows effective mitigation of the attack without losing functionality, only incurring a performance penalty on affected requests. |
|
content-security-policy |
base-uri ‘none’; object-src ‘none’; script-src ‘unsafe-eval’ ‘report-sample’ https://community.rstudio.com/logs/ https://community.rstudio.com/sidekiq/ https://community.rstudio.com/mini-profiler-resources/ https://community.rstudio.com/assets/ https://community.rstudio.com/brotli_asset/ https://community.rstudio.com/extra-locales/ https://community.rstudio.com/highlight-js/ https://community.rstudio.com/javascripts/ https://community.rstudio.com/plugins/ https://community.rstudio.com/theme-javascripts/ https://community.rstudio.com/svg-sprite/ https://www.google-analytics.com/analytics.js; worker-src ‘self’ blob: |
TRUE |
Read the reference http://www.w3.org/TR/CSP/ and set according to your case. This is not a easy job. |
|
content-type |
text/html; charset=utf-8 |
TRUE |
Properly configure their origin server to provide the correct Content-Type for a given representation. |
|
date |
Wed, 06 Mar 2019 12:27:24 GMT |
FALSE |
||
referrer-policy |
strict-origin-when-cross-origin |
NA |
NA |
NA |
server |
nginx |
TRUE |
An origin server SHOULD NOT generate a Server field containing needlessly fine-grained detail and SHOULD limit the addition of subproducts by third parties. |
|
strict-transport-security |
max-age=31536000 |
TRUE |
Please at least read this reference: https://www.owasp.org/index.php/HTTP_Strict_Transport_Security. |
|
vary |
Accept-Encoding |
FALSE |
||
x-content-type-options |
nosniff |
TRUE |
Always use the only defined value, “nosniff”. |
|
x-discourse-route |
list/latest |
NA |
NA |
NA |
x-download-options |
noopen |
NA |
NA |
NA |
x-frame-options |
SAMEORIGIN |
TRUE |
In 2009 and 2010, many browser vendors ([Microsoft-X-Frame-Options] and [Mozilla-X-Frame-Options]) introduced the use of a non-standard HTTP [RFC2616] header field “X-Frame-Options” to protect against clickjacking. Please check here https://www.owasp.org/index.php/Clickjacking_Defense_Cheat_Sheet what’s the best option for your case. |
|
x-permitted-cross-domain-policies |
none |
NA |
NA |
NA |
x-request-id |
4535cc47-f085-4bad-80ae-97aa50034956 |
NA |
NA |
NA |
x-runtime |
0.117140 |
NA |
NA |
NA |
x-xss-protection |
1; mode=block |
TRUE |
Use “X-XSS-Protection: 1; mode=block” whenever is possible (ref. http://blogs.msdn.com/b/ieinternals/archive/2011/01/31/controlling-the-internet-explorer-xss-filter-with-the-x-xss-protection-http-header.aspx). |
data(http_headers)
dplyr::glimpse(http_headers)
## Observations: 184
## Variables: 14
## $ header_field_name <chr> "A-IM", "Accept", "Accept-Additions", "Accept-Charset", "Accept-Datetime", "Accept-Encod…
## $ type_1 <chr> "Permanent", "Permanent", "Permanent", "Permanent", "Permanent", "Permanent", "Permanent…
## $ protocol <chr> "http", "http", "http", "http", "http", "http", "http", "http", "http", "http", "http", …
## $ status <chr> "", "standard", "", "standard", "informational", "standard", "", "standard", "", "standa…
## $ reference <chr> "https://tools.ietf.org/html/rfc3229#section-10.5.3", "https://tools.ietf.org/html/rfc72…
## $ type_2 <chr> "Request", "Request", "Request", "Request", "Request", "Request", "Request", "Request", …
## $ enable <lgl> FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, TRUE, TRUE, FALSE, TRUE, FALSE, …
## $ required <lgl> NA, NA, NA, NA, NA, NA, NA, NA, TRUE, TRUE, NA, TRUE, NA, NA, NA, TRUE, NA, NA, NA, NA, …
## $ https <lgl> NA, NA, NA, NA, NA, NA, NA, NA, TRUE, TRUE, NA, TRUE, NA, NA, NA, TRUE, NA, NA, NA, NA, …
## $ security_description <chr> "", "", "", "", "", "", "", "", "Sometimes an HTTP intermediary might try to detect viru…
## $ security_reference <chr> "", "", "", "", "", "", "", "", "https://tools.ietf.org/html/rfc5789#section-5", "https:…
## $ recommendations <chr> "", "", "", "", "", "", "", "", "Antivirus software scans for viruses or worms.", "Serve…
## $ cwe <chr> "", "", "", "", "", "", "", "", "CWE-509: Replicating Malicious Code (Virus or Worm)", "…
## $ cwe_url <chr> "\r", "\r", "\r", "\r", "\r", "\r", "\r", "\r", "https://cwe.mitre.org/data/definitions/…
Lang | # Files | (%) | LoC | (%) | Blank lines | (%) | # Lines | (%) |
---|---|---|---|---|---|---|---|---|
R | 11 | 0.92 | 186 | 0.9 | 37 | 0.57 | 85 | 0.63 |
Rmd | 1 | 0.08 | 21 | 0.1 | 28 | 0.43 | 50 | 0.37 |
Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms.