Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add missing options for curl_setopt() function #916

Open
wants to merge 45 commits into
base: master
Choose a base branch
from

Conversation

mt-omarov
Copy link

@mt-omarov mt-omarov commented Oct 14, 2023

Update KPHP curl_setopt() function

This PR extends the options of the curl_setopt() function:

  • primitive options of the curl_setopt() function up to the current version of php
  • support for non-standard CURLOPT_*FUNCTIONs options, as well as stream descriptor options

Requirements

libcurl version 8.4.0

Support for callable options

PHP supports wrappers for custom functions for parameters: CURLOPT_WRITEFUNCTION, CURLOPT_HEADERFUNCTION, CURLOPT_READFUNCTION, CURLOPT_PROGRESSFUNCTION, CURLOPT_XFERINFOFUNCTION.

Problem

To be able to pass callable objects to runtime and not override the logic of accepting mixed arguments, the curl_setopt() function must be overloaded.
However, overloading with a callable argument is not so simple: compiler optimizations prevent lambda expressions from being called from the runtime. The generated lambda expression code contains only the functionality that is explicitly called in the php code. Lambda expressions passed to curl_setopt() are called inside the runtime, so the compiler simply does not create methods to call the generated lambda objects => it will not be possible to call them in the runtime.

Solution

The key change for the overload operation is the rearrangement of the optimization step EarlyOptimizationF in compiler.cpp.

Next, to redirect the function call, use:

Logic

The lambda expressions that the user passes to curl_setopt() are stored in an object of the EasyContext class. Wrapper functions are defined in the runtime for each of the CURLOPT_*FUNCTIONs options:

Support for stream options

Added support for options: CURLOPT_FILE, CURLOPT_INFILE, CURLOPT_WRITEHEADER.
These options redirect the output of information in the CURLOPT_*FUNCTIONs to the installed stream descriptors.
It was necessary to add:

Support for missing primitive options

  • added bitmasks for CURLOPT_PROTOCOLS;
  • added bitmasks for CURLOPT_SSH_AUTH_TYPES;
  • added bitmasks for CURLAUTH options;
  • added bitmasks for SSL options:
    unsupported flags are not allowed to be transmitted: CURLSSLOPT_NO_REVOKE, CURLSSLOPT_REVOKE_BEST_EFFORT, CURLSSLOPT_AUTO_CLIENT_CERT

The full list of added options is located in the curl-options.md

Tests

@astrophysik astrophysik self-requested a review October 17, 2023 13:07
@astrophysik
Copy link
Contributor

Is this pr ready for review?

@mt-omarov
Copy link
Author

Is this pr ready for review?

not yet

@Danil42Russia
Copy link
Contributor

Is this pr ready for review?

not yet

Convert it to Draft please

@andarut andarut marked this pull request as draft October 29, 2023 17:58
@andarut andarut marked this pull request as ready for review November 1, 2023 10:49
@andarut andarut marked this pull request as draft November 1, 2023 10:49
@mt-omarov mt-omarov force-pushed the curl branch 2 times, most recently from 2a1c383 to cbe6365 Compare November 1, 2023 11:05
…setopt, expanded curl_getinfo by adding CURLINFO_CERTINFO logic
…AL, PATH_AS_IS, PIPEWAIT, SASL_IR, CONNECT_TO, PROXYHEADER
…FO_CERTINFO option. Added curl_setopt options: SSL_VERIFYSTATUS, NORPOGRESS, NOSIGNAL, PATH_AS_IS, PIPEWAIT, SASL_IR, CONNECT_TO, PROXYHEADER
…_HAPPY_EYEBALLS_TIMEOUT_MS, CURLOPT_HEADEROPT, CURLOPT_POSTREDIR. Defined flags for the logic of headeropt and postredir options
…PY_EYEBALLS_TIMEOUT_MS, CURLOPT_HEADEROPT, CURLOPT_POSTREDIR. Created new option_setter functions for the headeropt and postredir options.
… CURLOPT_DEFAULT_PROTOCOL, CURLOPT_DNS_INTERFACE. Added curl_version() function.
…OXY_SSL_CIPHER_LIST, CURLOPT_PROXY_TLS13_CIPHERS, CURLOPT_PROXY_SSLKEY, etc. Removed not supported CURLOPT_DNS_INTERFACE option.

(op_func_call {"curl_setopt"} curl_handle option:(op_int_const {"210004"}) cb)
=> (op_func_call {"_curl_setopt_fn_xferinfo"} curl_handle option cb)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The logic of overloading a function depending on the value of an option

This construction allows the compiler to redirect the call to the curl_setopt function to the specified one if the value of the option parameter is 21000.

Syntax:

  • (op_func_call {"<name of the original function>"} first_arg_name second_arg_name:(op_int_const {"<value>"}) third_arg_name)
  • (op_func_call {"<name of the new function>"} args)

The construction above redirects the function call <name of the original function> to <name of the new function> if the parameter <second_arg_name> is equal to the value <value>

struct {
curl::on_xferinfo_callable callable{NULL};
} xferinfo_handler;

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Logic for storing user-defined functions for callable options of curl_setopt()

In the EasyContext class was added fields, which store callable objects and additional information about the mode of operation of functions that are passed to libcurl.

Arguments information:

  • callable: a field for user-defined function
  • method: defines the mode of operation of the final function
  • stream: pointer to the stream descriptor (if any)

Methods:

  • KPHP_CURL_STDOUT: the standard output of information for curl_write() and curl_write_header();
  • KPHP_CURL_RETURN: a special mode of operation for curl_write(), in which information is returned from the function;
  • KPHP_CURL_USER: the mode of operation in which a user-defined function is called (from callable field)
  • KPHP_CURL_IGNORE: a special mode of operation for curl_write_header(), in which a length of the information is returned from the function;
  • KPHP_CURL_DIRECT: the standard output of information for curl_read();
  • KPHP_CURL_FILE: the mode of operation for curl_write() and curl_write_header() for writing information to a stream descriptor from the stream field

if (auto *easy_context = get_context<EasyContext>(easy_id)) {
switch (option) {
case CURLSETOPT_HEADERFUNCTION:
easy_context->write_header_handler.callable = callable;
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The process of installing the user's callable objects when calling curl_setopt()

After successful redirection by the compiler of the curl_setopt() function to the function from kphp_internal.txt the callable object is saved to the appropriate handler. In this case, the mode of operation of the corresponding curl function (method field) is set to the value KPHP_CURL_USER.

@mt-omarov mt-omarov changed the title Add missing options for curl_setopt() function Add missing options for curl_setopt() function Jan 26, 2024
@mt-omarov mt-omarov marked this pull request as ready for review January 26, 2024 18:48
@mt-omarov mt-omarov force-pushed the curl branch 6 times, most recently from e67b8c3 to 4ad9d86 Compare February 26, 2024 13:55
builtin-functions/_funcions.txt:
	Added an alias CURLOPT_WRITEDATA for the option CURLOPT_FILE.
runtime/curl.h:
	Fixed callable type names (<callable_type> -> on_<callable_type>), moved it to namespace.
runtime/curl.cpp:
	1. Brought to a general appearance all fields of callable objects of the Easy_Context class: struct {...} <callable_name>_handle.
	2. Added an additional null value check to the stream_option_setter function.
	3. Deleted the incorrect installation of the method of operation of the curl_read function when installing the CURLOPT_INFILE option.
	4. Replaced the curl error code CURLE_WRITE_ERROR with -1 for errors in the stream_option_setter function.
	5. Fixed a bug in the curl_read function: copying to a buffer variable is done via calls to $fgets() and memcpy().
Refactored class Process:
	1. Deleted unnecessary function run_cmd().
	2. Fixed the definition of the process id using the lsof call relative to the listening port.
	3. Added exception throwing and corrected syntax.
Added a test for the CURLOPT_WRITEHEADER option:
	1. It works by analogy with the test for CURLOPT_FILE.
	2. The response headers are written to the file, but kphp returns more information => only the beginning of the file is checked.
All tests now take 60 ms to start the server.
…RESSFUNCTION. Refactored tests for stream-options.

14_curl_setopt_streams.php:
	1. Moved common variables to the global scope.
	2. Increased delays for creating and closing the server: from 60 to 200 ms.
17_curl_setopt_callables.php:
	1. Option tests were created by analogy with stream tests.
	2. Port values for php and kphp differ from stream tests to avoid intersections.
	3. Testing the PROGRESSFUNCTION option uses the CURLOPT_FILE stream option and raises the server, which sends the locally created document to the client when the request is received.
…s in stream-tests. Rewrited the logic of curl_read function in runtime/curl.cpp.

curl_read() in runtime/curl.cpp:
	Previously, the result of calling a custom function was missed, now the returned string is written to memory via memcpy().
testing CURLOPT_READFUNCTION:
	1. The test is written by analogy with the test of the CURLOPT_INFILE option, which sends data to the server.
	2. At the moment, the user-defined function is required to return exactly the string.
…setter() function

In the f$curl_setopt() function with options for stream descriptors, it is important to check the opening mode of these descriptors.
This is only possible by accessing a static variable from files.cpp. In order to solve the access problem, a new get_stream_mode() function has been added to files.h and cpp.

The stream_option_setter() function has been fixed:
	- it is allowed to pass null as an argument for unsetting variables;
	- checking for the file descriptor opening mode is now performed via get_stream_mode();
	- the set_option_safe() call has been moved to the beginning of the function to save the error_code field in the easy_context
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants