Linux Calling...: cURL HTTP Pipelining

Thursday, September 26, 2013

cURL HTTP Pipelining

One of my friend asked me if there was a tool to send pipelined http request from linux command line. Thanks to cURL libcurl, I developed a small tool to do the same. Have put in some basic validations and some restrictions like the maximum no of pipeline request is 32 etc

/* 
 * Ahamed 
 * Using cURL libcurl
 *
 * For sending pipelined http requests
 * Usage : ./send_pipelined_http <no of pipelined requests> <http://x.x.x.x>
 * Some basic validations added.
*/

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <curl/curl.h>

static void usage();

#define ERR_LEN 512
#define MIN_URL_LEN 14
#define MAX_URL_LEN 32 
#define MIN_PIPELINE_REQ 1
#define MAX_PIPELINE_REQ 32
#define err(x, y) {printf("\nError: %s...\nExiting...\n", x); if(y)exit(0); usage();}

static unsigned int pipeline_req;
static char url[MAX_URL_LEN];
static char err_str[ERR_LEN];

static void handle_curl_error(CURLcode ret, char *func)
{
  if(ret != CURLE_OK){
    fprintf(stderr, "%s() failed: %s\n", func,
            curl_multi_strerror(ret));
  }
  return;
}

static void usage()
{
  printf("\nDeveloped by Ahamed using cURL API via libcurl\n");
  printf("Usage : ./send_pipelined_http <no of req> <http://x.x.x.x>\n");
  printf("        Pipeline Request - Min %d : Max : %d\n", 
                  MIN_PIPELINE_REQ, MAX_PIPELINE_REQ);
  printf("        Hostname not supported yet. Please provide IP address.\n");
  printf("        URL - Min Length %d : Max Length : %d\n\n", 
                  MIN_URL_LEN, MAX_URL_LEN);
  exit(0);
}

static void parse_args(int argc, char **argv)
{

  if(argc <= 2)
    err("Insufficient arguments", 0);

  if(!argv)
    err("Something unexpected happened", 1);

  pipeline_req = atol(argv[1]);
  if(pipeline_req > MAX_PIPELINE_REQ || pipeline_req < MIN_PIPELINE_REQ)
    err("Invalid Pipeline Request", 0);
  
  if(strlen(argv[2]) > MAX_URL_LEN || strlen(argv[2]) < MIN_URL_LEN)
    err("Invalid URL length", 0);

  strncpy(url, argv[2], MAX_URL_LEN);
  
  return;
}
 
int main(int argc, char **argv)
{

  int i=0;
  parse_args(argc, argv);

  CURLM *m_curl;
  CURLMcode res;

  m_curl = curl_multi_init();
  curl_multi_setopt(m_curl, CURLMOPT_PIPELINING, 1L);

  CURL *curl[MAX_PIPELINE_REQ];
  for(i=0; i<pipeline_req; i++){

    curl[i] = curl_easy_init();
    if(!curl[i])
      err("Something went wrong", 0);

    res = curl_easy_setopt(curl[i], CURLOPT_URL, url);
    handle_curl_error(res, "curl_easy_setopt");

    res = curl_easy_setopt(curl[i], CURLOPT_FOLLOWLOCATION, 1L);
    handle_curl_error(res, "curl_easy_setopt");

    res = curl_multi_add_handle(m_curl, curl[i]);
    handle_curl_error(res, "curl_multi_add_handle");
  }

  int ret = 1;
  do {
    res = curl_multi_perform(m_curl, &ret);
  } while(ret);

  handle_curl_error(res, (char*)__FUNCTION__);

  for(i=0; i<pipeline_req; i++){
    curl_multi_remove_handle(m_curl, curl[i]); 
    curl_easy_cleanup(curl[i]);
  }
  curl_multi_cleanup(m_curl);

  return 0;
}

root@Imperfecto_maximuS> gcc send_pipelined_http.c -lcurl -o send_pipelined_http
root@Imperfecto_maximuS> ./send_pipelined_http

Error: Insufficient arguments...
Exiting...

Developed by Ahamed(ahamed.en@gmail.com) using cURL API via libcurl
Usage : ./send_pipelined_http <no of req> <http://x.x.x.x>
        Pipeline Request - Min 1 : Max : 32
        Hostname not supported yet. Please provide IP address.
        URL - Min Length 14 : Max Length : 32

root@Imperfecto_maximuS>  ./send_pipelined_http 3 http://11.1.1.99
<html><body><h1>This is sample page</h1>\0<p>for testing created...</p></body></html>
<html><body><h1>This is sample page</h1>\0<p>for testing created...</p></body></html>
<html><body><h1>This is sample page</h1>\0<p>for testing created...</p></body></html>

16 comments:

VenkatOctober 9, 2013 at 6:41 PM
Hi, Thanks for the program, I am getting the below error when executing. I am using gcc version 4.1.2 20080704 (Red Hat 4.1.2-46)
and CURL version curl 7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5

[build@iforge1 venkat]$ gcc pipeline.c -lcurl -o pipeline
pipeline.c: In function âmainâ:
pipeline.c:80: error: âCURLMOPT_PIPELININGâ undeclared (first use in this function)
pipeline.c:80: error: (Each undeclared identifier is reported only once
pipeline.c:80: error: for each function it appears in.)

ReplyDelete
Replies
TechnoZombieJanuary 13, 2014 at 7:10 PM
Its not working can you pls help me getting below error
send_pipelined_http.c:10:19: stdio.h: No such file or directory
send_pipelined_http.c:11:20: stdlib.h: No such file or directory
send_pipelined_http.c:12:20: string.h: No such file or directory
send_pipelined_http.c:13:23: curl/curl.h: No such file or directory
send_pipelined_http.c:28: error: parse error before "ret"
send_pipelined_http.c: In function `handle_curl_error':
send_pipelined_http.c:30: error: `ret' undeclared (first use in this function)
send_pipelined_http.c:30: error: (Each undeclared identifier is reported only once
send_pipelined_http.c:30: error: for each function it appears in.)
send_pipelined_http.c:30: error: `CURLE_OK' undeclared (first use in this function)
send_pipelined_http.c:31: error: `stderr' undeclared (first use in this function)
send_pipelined_http.c:31: error: `func' undeclared (first use in this function)
send_pipelined_http.c: In function `main':
send_pipelined_http.c:76: error: `CURLM' undeclared (first use in this function)
send_pipelined_http.c:76: error: `m_curl' undeclared (first use in this function)
send_pipelined_http.c:77: error: `CURLMcode' undeclared (first use in this function)
send_pipelined_http.c:77: error: parse error before "res"
send_pipelined_http.c:80: error: `CURLMOPT_PIPELINING' undeclared (first use in this function)
send_pipelined_http.c:82: error: `CURL' undeclared (first use in this function)
send_pipelined_http.c:82: error: `curl' undeclared (first use in this function)
send_pipelined_http.c:89: error: `res' undeclared (first use in this function)
send_pipelined_http.c:89: error: `CURLOPT_URL' undeclared (first use in this function)
send_pipelined_http.c:92: error: `CURLOPT_FOLLOWLOCATION' undeclared (first use in this function)
ReplyDelete
Replies
AnonymousMarch 28, 2014 at 4:18 PM
Requests are not pipelined.
ReplyDelete
Replies
UnknownJune 22, 2018 at 10:14 PM
I compiled this code today on FreeBSD using the latest libCurl (7.60.0) and I see it making three connections to the server, and then looping forever in the "res = curl_multi_perform(m_curl, &ret);" loop with ret == 2.

Murf
ReplyDelete
Replies

Add comment