In the first part of this series, I explained my fuzzing workflow and covered some of the custom mutators I’ve built for fuzzing Apache HTTP. In the second part, I explained how to build custom ASAN interceptors in order to catch memory bugs when custom memory pools are used.
In this third and last part, I’ll share the results of my research on Apache HTTP server, and I’ll show some of the vulnerabilities that I’ve found.
So, let’s get to it!
NULL dereference in session_identity_decode
This bug can be triggered setting a cookie with a NULL key and value:
In the example above, you can see that in the first position of the cookies there is a session key and a choco value. In the second position, we can find the admin-user key and the number 2 as a value. However, in the third position there is an empty key and value pair.
What’s the problem here? Well, if you look at the following code snippet, you can see two calls to apr_strtok, to extract the first and the second string (key and value):
const char *psep = "=";
char *key = apr_strtok(pair, psep, &plast);
char *val = apr_strtok(NULL, psep, &plast);
Let’s see now what happens in the apr_strtok function when the first argument is NULL:
APR_DECLARE(char *) apr_strtok(char *str, const char *sep, char **last)
{
char *token;
if (!str)
str = *last;
while (*str && strchr(sep, *str))
++str;
You can see in this code snippet how the while loop tries to dereference the first function argument (str pointer). So, if this first argument is NULL, it will trigger a NULL dereference bug. Also,this is exactly what happens with the following statement char *val = apr_strtok(NULL, psep, &plast); when the previous key value is also null.
In order to exploit this bug, mod_session needs to be enabled. This vulnerability can lead to a denial of service at the child level, affecting the other threads in the same process.
Off-by-one (stack-based) in check_nonce
In order to exploit this bug, the mod_auth_digest module should be enabled, and the application has to be using the DIGEST authentication.
For triggering this bug, we need to assign a specific set of values to the nonce field as follows:
GET http://127.0.0.1/i?proxy=yes HTTP/1.1
Host: foo.example
Accept: */*
Authorization: Digest username="2",
realm="private area",
nonce="d2hhdGFzdXJwcmlzZXhkeGR4ZHhkeGR4ZHhkeGR4ZHhkeGR4ZA==",
uri="http://127.0.0.1:80/i?proxy=yes",
qop=auth,
nc=00000001,
cnonce="0a4f113b",
response="53849ce65ba787cd0a07a272ece3bba6",
opaque="5ccc069c403ebaf9f0171e9517f40e41"
As you can see, the nonce field is a BASE64 value. In order to decode this value, the check_nonce function does a call to:
apr_base64_decode_binary(nonce_time.arr, resp->nonce)
where nonce_time.arr is a local array of size 8. Let’s see the code of the apr_base64_decode_binary function:
APR_DECLARE(int) apr_base64_decode_binary(unsigned char *bufplain, const char *bufcoded)
{
int nbytesdecoded;
register const unsigned char *bufin;
register unsigned char *bufout;
register apr_size_t nprbytes;
bufin = (const unsigned char *) bufcoded;
while (pr2six[*(bufin++)] <= 63);
nprbytes = (bufin - (const unsigned char *) bufcoded) - 1;
nbytesdecoded = (((int)nprbytes +3) / 4) * 3;
bufout = (unsigned char *) bufplain;
bufin = (const unsigned char *) bufcoded;
while (nprbytes > 4) {
*(bufout++) =
(unsigned char) (pr2six[*bufin] << 2 | pr2six[bufin[1]] >> 4);
*(bufout++) =
(unsigned char) (pr2six[bufin[1]] << 4 | pr2six[bufin[2]] >> 2);
*(bufout++) =
(unsigned char) (pr2six[bufin[2]] << 6 | pr2six[bufin[3]]);
bufin += 4;
nprbytes -= 4;
}
if (nprbytes > 1) {
*(bufout++) =
(unsigned char) (pr2six[*bufin] << 2 | pr2six[bufin[1]] >> 4);
}
if (nprbytes > 2) {
*(bufout++) =
(unsigned char) (pr2six[bufin[1]] << 4 | pr2six[bufin[2]] >> 2);
}
if (nprbytes > 3) {
*(bufout++) =
(unsigned char) (pr2six[bufin[2]] << 6 | pr2six[bufin[3]]);
}
Under normal circumstances, the nprbytes variable will give 11 as a result, and the while loop will be executed two times, writing a total of 8 bytes into the bufplain array (6 + 2).
However, if the date format is wrong, the calculation of the nprbytes variable can give 12 as a result. So in these cases, the while loop will be executed three times and 9 bytes will be written into the bufplain array. Consequently, the program is writing 1 byte outside of the boundaries of the local array nonce_time.arr, overwriting 1 byte in the program stack (aka off-by-one).
Use-after-free in cleanup_tables
Here we have a use-after-free (UAF) in the cleanup_tables function. Let’s take a look at the code:
static apr_status_t cleanup_tables(void *not_used)
{
ap_log_error(APLOG_MARK, APLOG_INFO, 0, NULL, APLOGNO(01756)
"cleaning up shared memory");
if (client_rmm) {
apr_rmm_destroy(client_rmm);
client_rmm = NULL;
}
if (client_shm) {
apr_shm_destroy(client_shm);
client_shm = NULL;
}
You can see this function calls the apr_rmm_destroy function in order to free the client_rmm memory block. Yet the problem here is that, under certain circumstances, this memory block could have been previously freed by the apr_allocator_destroy function (not shown in this code snippet).
So, the program is trying to access an address that is no longer valid, leading to a use-after-free vulnerability. It’s important to mention that this vulnerability can only be triggered in the ONE_PROCESS mode.
OOB-write (heap-based) in ap_escape_quotes
In this case, we have a heap out-of-bounds write affecting the ap_escape_quotes function. This function escapes any quotes in the given input string. The origin of this bug is a calculation mismatch, between the length of the input string and the size of the “malloced” outstring buffer.
In the following code snippet, you can see the code that calculates the length of the input string:
while (*inchr != '\0'){
newlen++;
if (*inchr == '"') {
newlen++;
}
if ((*inchr == '\\') && (inchr[1] != '\0')) {
inchr++;
newlen++;
}
inchr++;
}
outstring = apr_palloc(p, newlen + 1);
In this second code snippet, you can see the code that calculates the size of outstring:
while (*inchr != '\0') {
if ((*inchr == '\\') && (inchr[1] != '\0')) {
*outchr++ = *inchr++;
*outchr++ = *inchr++;
}
if (*inchr == '"') {
*outchr++ = '\\';
}
if (*inchr != '\0') {
*outchr++ = *inchr++;
}
}
*outchr = '\0';
return outstring;
As you can see, it uses a different logic for this second size calculation. As a result, if we provide a malicious input to the ap_escape_quotes function, it is possible to write outside the bounds of the outchr array.
This bug was previously reported by Google OSS-Fuzz, just a few days before I found it.
Race condition leading to UAF
Now, I’m going to explain something totally different. In this case, the bug is a race condition leading to use-after-free and affecting the Apache Core.
During my fuzzing work, I found multiple non-reproducible UAF crashes. After looking into it more deeply, I discovered a kind of race condition between calls to apr_allocator_destroy and allocator_alloc. All the signs suggested that these functions might not be thread safe in concurrent scenarios. This could lead to a corruption of some nodes of the memory pool and, occasionally, the program tries to release a block that is already present in the free pool. This bug shares some similarities with the bug I reported in ProFTPD (CVE-2020-9273), a year ago.
Here you can see an example ASAN trace:
==106820==ERROR: AddressSanitizer: heap-use-after-free on address 0x625000091100 at pc 0x7ffff7d2ff4d bp 0x7fffffffd800 sp 0x7fffffffd7f8
READ of size 8 at 0x625000091100 thread T0
#0 0x7ffff7d2ff4c in apr_allocator_destroy /home/antonio/Downloads/httpd-trunk/srclib/apr/memory/unix/apr_pools.c:197:26
#1 0x7ffff7d3306c in apr_pool_terminate /home/antonio/Downloads/httpd-trunk/srclib/apr/memory/unix/apr_pools.c:756:5
#2 0x7ffff77aeba6 in __run_exit_handlers /build/glibc-5mDdLG/glibc-2.30/stdlib/exit.c:108:8
#3 0x7ffff77aed5f in exit /build/glibc-5mDdLG/glibc-2.30/stdlib/exit.c:139:3
#4 0x5b1ae8 in clean_child_exit /home/antonio/Downloads/httpd-trunk/server/mpm/event/event.c:777:5
#5 0x5b19a5 in child_main /home/antonio/Downloads/httpd-trunk/server/mpm/event/event.c:2957:5
#6 0x5afa7b in make_child /home/antonio/Downloads/httpd-trunk/server/mpm/event/event.c:2981:9
#7 0x5af005 in startup_children /home/antonio/Downloads/httpd-trunk/server/mpm/event/event.c:3046:13
#8 0x5a74c1 in event_run /home/antonio/Downloads/httpd-trunk/server/mpm/event/event.c:3407:9
#9 0x6212b1 in ap_run_mpm /home/antonio/Downloads/httpd-trunk/server/mpm_common.c:100:1
#10 0x5e67e6 in main /home/antonio/Downloads/httpd-trunk/server/main.c:891:14
#11 0x7ffff778c1e2 in __libc_start_main /build/glibc-5mDdLG/glibc-2.30/csu/../csu/libc-start.c:308:16
#12 0x44da7d in _start ??:0:0
This is not a new problem. Similar issues were reported by Hanno Böck (@hanno) in 2018. You can check Hanno’s previous reports here.
Minor bugs
During my fuzzing session, I found some other minor bugs, and I would like to show you one of them: an integer overflow in the Session_Identity_Decode function. It’s not a dangerous bug, but I think that it can be interesting to show an example of how trivial it is to trigger this bug.
So, take a look at the following example in which we send a LOCK WebDav request to MOD_DAV with a large TimeOut value (Second-41000000004100000000):
LOCK /dav/c HTTP/1.1
Host: 127.0.0.1
Timeout: Second-41000000004100000000
Content-Type: text/xml; charset="utf-8"
Content-Length: XXX
Authorization: Basic Mjoz
<?xml version="1.0" encoding="utf-8" ?>
<d:lockinfo xmlns:d="DAV:">
In the following code snippet you can see the statement:
return now + expires;
where there is an addition of two (32-bit) integer values, that will be stored in another integer variable. So if these values are big enough, we can overflow the returned value.
while ((val = ap_getword_white(r->pool, &timeout))
if (!strncmp(val, "Infinite", 8)) {
return DAV_TIMEOUT_INFINITE;
}
if (!strncmp(val, "Second-", 7)) {
val += 7;
expires = atol(val);
now = time(NULL);
return now + expires;
}
}
Since this bug is triggered with a LOCK request, the MOD_DAV module should be enabled in order to be triggered.
Conclusions
While Apache HTTP security has been extensively studied by researchers, based on recently disclosed vulnerabilities involving path traversals and file disclosures (CVE-2021-41773 and CVE-2021-42013), it is clear that there is still room for discovering new critical vulnerabilities.
With this research, I wanted to make my own contributions to improve the security of the Apache HTTP server, and show that it is possible to use fuzzing for finding vulnerabilities in one of the most used open source software out there. At the same time, I hope that I was able to share all the knowledge I learned with you.
What next?
With this third part of my “fuzzing Apache” research, I concluded the “Fuzzing sockets” series. You can find a summary of all my previous posts here:
- Fuzzing sockets, Part 1: FTP servers
- Fuzzing sockets, Part 2: FreeRDP
- Fuzzing sockets: Apache HTTP, Part 1: Mutations
- Fuzzing sockets: Apache HTTP, Part 2: Custom Interceptors
In my next blog entry, I’ll start a new series focused on fuzzing Javascript engines. Stay tuned!
Need more information?
I’ve used the following resources in this blog post:
- http://httpd.apache.org/docs/
- http://www.apachetutor.org/dev/pools