Skip to content

Comments

allocate async resolv state#20572

Draft
icing wants to merge 5 commits intocurl:masterfrom
icing:state-async-allocated
Draft

allocate async resolv state#20572
icing wants to merge 5 commits intocurl:masterfrom
icing:state-async-allocated

Conversation

@icing
Copy link
Contributor

@icing icing commented Feb 12, 2026

make async resolve state allocated

Turn the fixed easy handle member async into an allocated struct. We allocated the contained hostname already, so fold that into the struct itself and make the easy handle a little smaller.

stacked on #20570

@github-actions github-actions bot added the tests label Feb 12, 2026
@icing icing force-pushed the state-async-allocated branch 4 times, most recently from 17a71b9 to 74904c7 Compare February 13, 2026 12:12
icing added a commit to icing/curl that referenced this pull request Feb 13, 2026
For async, threaded resolving use a thread queue attached to
the multi handle for processing. When the multi has a WAKEUP
socket pair, use that for notifying the multi about results.

Starts up a maximum number of threads which wind down again
after an idle time.

Inidividual transfer no longer have their own threads or
socket pairs for name resolving.

stacked on curl#20572
icing added a commit to icing/curl that referenced this pull request Feb 19, 2026
For async, threaded resolving use a thread queue attached to
the multi handle for processing. When the multi has a WAKEUP
socket pair, use that for notifying the multi about results.

Starts up a maximum number of threads which wind down again
after an idle time.

Inidividual transfer no longer have their own threads or
socket pairs for name resolving.

stacked on curl#20572
    Add curl_thrdpool and associated methods for managing a pool of threads that can resize between min/max number of threads waiting for "work" to "process".

    Threads above the min number will exit after an idle timeout. A pool can be waited for to become idle, e.g. having processed everything. A pool can be destroyed either by joining all threads or detaching them. In the later case the pool's memory will be released when the last thread exits.

    Add unit3217 test to verify thread pools operation.

    Other adjustments:

    curl_setup.h: define USE_THREADS when either POSIX or WINDOWS threads are configured for simpler ifdefs
    memdebug.c: in atexit handle, inspect global var under lock. Otherwise thread sanitizer will report unsynchronized data access
    thread queue

    Add curl_thrdq and associated methods for a "processing queue" where one can send "items" for processing and receiving them afterwards. The thread queue uses a thread pool to perform the processing.

    Add unit3218 test to verify the thread queue operation.
        Add the multi wakeup socket (when available) to the pollset
        of the admin handle *when* transfers are alive. This markes
        the admin handle as "dirty" on socket events and lets it run.

        Without this, event based processing never sees the wakeup socket
        and never acts on it. I assume the wakeup socket never worked for
        event based processing.

        What is this good for?

        In a future PR, based on this one, we can trigger async resolver
        results to use the multi's wakeup socket and save the transfer
        based wakeup sockets. This also solves lifetime problems for such
        sockets when a transfer is destroyed early, as the multi wakeup
        system remains unchanged during the multi's lifetime.
        Give the dnscache implementation its own source file, as
        hostip.c became unwieldy.

        have Curl_dnscache_get() return a CURLcode for negative entries
        fix include for unit1305.c
        - remove `struct Curl_dns_entry *dns` entry
        - remove `done` BIT
        - make port an uint16_t
        - make ip_version an uint8_t
        - rename `Curl_async_is_resolved()` to `Curl_async_take_result()`,
          returning CURLE_AGAIN while waiting on the result
        - remove `Curl_once_resolved()`
        Turn the fixed easy handle member `async` into an allocated struct.
        We allocated the contained hostname already, so fold that into the
        struct itself and make the easy handle a little smaller.
@icing icing force-pushed the state-async-allocated branch from 74904c7 to 5827485 Compare February 19, 2026 14:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Development

Successfully merging this pull request may close these issues.

1 participant