Главная » Языки программирования

Curl no html output

На чтение 9 мин
Содержание
  1. Curl — Display Request and Response Headers
  2. Display only Response Headers in curl
  3. You May Also Enjoy
  4. Как заставить cURL выдавать результат, а не выводить на экран?
  5. Curl cheatsheet
  6. #Getting Started
  7. #Introduction
  8. #Options
  9. #Request
  10. #data
  11. #Header information Headers
  12. #SSL
  13. #Install
  14. #Example
  15. #CURL GET/HEAD
  16. #Multiple file upload
  17. #Prettify json output for curl response
  18. #CURL POST
  19. #CURL script install rvm
  20. #CURL Advanced
  21. #Check website response time
  22. #Use Curl to check if a remote resource is available
  23. #Downloading file
  24. #Download the file, save the file without changing its name
  25. #continue partial download
  26. #Download files from multiple domains
  27. #Download a series of files
  28. #Redirect output to file
  29. #Basic Authentication
  30. #Write to file instead of stdout
  31. #Download header information
  32. #Write output to a file named remote_file
  33. #Execute remote script
  34. #Configuration file

Curl — Display Request and Response Headers

In default mode, curl doesn’t display request or response headers, only displaying the HTML contents.

To display both request and response headers, we can use the verbose mode curl -v or curl -verbose . In the resulting output:

  • The lines beginning with > indicate request headers.
  • The lines beginning with < indicate response headers.
  • (The lines beginning with * indicate additional information.)

Let’s try curl -v http://codeahoy.com to print request and response headers. In the image below, the request and response headers are highlighted.

curl output showing request and repsonse headers

curl -v http://codeahoy.com * Trying 2606:4700:3032::ac43:d1bc. * TCP_NODELAY set * Connected to codeahoy.com (2606:4700:3032::ac43:d1bc) port 80 (#0) > GET / HTTP/1.1 > Host: codeahoy.com > User-Agent: curl/7.64.1 > Accept: */* > < HTTP/1.1 301 Moved Permanently < Date: Sun, 22 Aug 2021 18:57:16 GMT < Transfer-Encoding: chunked < Connection: keep-alive < Cache-Control: max-age=3600 < Expires: Sun, 22 Aug 2021 19:57:16 GMT < Location: https://codeahoy.com/ < Report-To: "endpoints":["url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=s7X0EJLhIVF8i%2Bkkxxzl1URcpPF65SmKK%2BRthSOgOcbA9xnyChBwSpVYL3eIjk369OlW52rawLwH9DR268EanSOrK9QAMTc7aQ35%2Bntje3X7QSFjeiXLmI2P3JkiRZMl1XqA%2Bm%2BYEaSySFM%3D">],"group":"cf-nel","max_age":604800> < NEL: "success_fraction":0,"report_to":"cf-nel","max_age":604800> < Server: cloudflare < CF-RAY: 682e536f49440d44-LAX < alt-svc: h3-27=":443"; ma=86400, h3-28=":443"; ma=86400, h3-29=":443"; ma=86400, h3=":443"; ma=86400 < * Connection #0 to host codeahoy.com left intact * Closing connection 0

Display only Response Headers in curl

If you want to display only the response headers, you can use the —head flag.

curl --head http://codeahoy.com HTTP/1.1 301 Moved Permanently Date: Sun, 22 Aug 2021 19:14:41 GMT Connection: keep-alive Cache-Control: max-age=3600 Expires: Sun, 22 Aug 2021 20:14:41 GMT Location: https://codeahoy.com/ Report-To: "endpoints":["url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=FJzR6TegRQYBvzipX6RglYE4NgeOvqiNS0%2Faf0Z7pLe0YDku7Gk7xvKgB8RRpvF9vAO81saiqA5PDCoY2Fzz%2BqSY9yQsbOdKjmFVVdYK4ccyYFpoKOoErreL9cqnXFo36%2FYaZcDFDheb%2FU4%3D">],"group":"cf-nel","max_age":604800> NEL: "success_fraction":0,"report_to":"cf-nel","max_age":604800> Server: cloudflare CF-RAY: 682e6cf26fe7094c-SEA alt-svc: h3-27=":443"; ma=86400, h3-28=":443"; ma=86400, h3-29=":443"; ma=86400, h3=":443"; ma=86400

Here’s an alternate way to —head for displaying response headers:

curl -s -D - -o /dev/null http://codeahoy.com
  • -s hides the progress bar
  • -D — dump headers to stdout indicated by —
  • -o /dev/null send output (HTML) to /dev/null essentially ignoring it
curl -s -D - -o /dev/null http://codeahoy.com HTTP/1.1 301 Moved Permanently Date: Sun, 22 Aug 2021 19:10:49 GMT Transfer-Encoding: chunked Connection: keep-alive Cache-Control: max-age=3600 Expires: Sun, 22 Aug 2021 20:10:49 GMT Location: https://codeahoy.com/ Report-To: "endpoints":["url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=7QSrtkFxzkNpCbvMNNqdiuS%2B8II41I%2F7AJRdHwqV49DiyuAyYgTSiV0ceaXx2Es0SJ1XcLmdz9xd6lkQqgl0oe7npIE3QokSgpf86ueyosfjFWHVJcCg5OpIngOkdb12NKz5hOTDbxNTsKg%3D">],"group":"cf-nel","max_age":604800> NEL: "success_fraction":0,"report_to":"cf-nel","max_age":604800> Server: cloudflare CF-RAY: 682e67499a52088d-SEA alt-svc: h3-27=":443"; ma=86400, h3-28=":443"; ma=86400, h3-29=":443"; ma=86400, h3=":443"; ma=86400

You May Also Enjoy

If you like this post, please share using the buttons above. It will help CodeAhoy grow and add new content. Thank you!

Источник

Как заставить cURL выдавать результат, а не выводить на экран?

С удивлением обнаружил, что обычный curl_exec выводит результат на экран, не смотря на CURLOPT_RETURNTRANSFER.

public function getHTML($url)< if(!isset($this->curl) || is_null($this->curl)) < $this->init(); > curl_setopt($this->curl, CURLOPT_URL, $url); $page = curl_exec($this->curl); $this->close(); return $page; > protected function init()< $this->curl = curl_init(); $this->setOptions(); curl_setopt_array($this->curl, $this->options); > private function setOptions()< if(empty($this->options)) < $this->options = array( CURLOPT_AUTOREFERER => true, CURLOPT_COOKIESESSION => false, CURLOPT_FOLLOWLOCATION => true, CURLOPT_HTTPGET => true, CURLOPT_RETURNTRANSFER => true, CURLOPT_BINARYTRANSFER => true, CURLOPT_ENCODING => 'gzip,deflate,sdch', CURLOPT_HTTPHEADER => array( 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'Accept-Charset: windows-1251,utf-8;q=0.7,*;q=0.3', 'Accept-Language: ru-RU,ru;q=0.8,en-US;q=0.6,en;q=0.4', 'Cache-Control: max-age=0' ), CURLOPT_USERAGENT => 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_2) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.75 Safari/535.7' ); if($this->cookie)< $this->options[CURLOPT_COOKIEFILE] = "cookie.txt"; $this->options[CURLOPT_COOKIEJAR] = "cookie.txt"; > > >

Думать было некода. Я вставил такой костыль:

ob_start(); curl_exec($this->curl); $this->close(); $page = ob_get_contents(); ob_clean();

Но при переносе проекта на боевой сервер оказалось, что там curl_exec уже не выводит на экран, а снова нужно старую версию включать.

Вопроса два:
Как сделать что бы работало и там и там?
Что я делал не так?

P.S. Версии php
на первом сервере:

PHP 5.5.9-1ubuntu4.11 (cli) (built: Jul 2 2015 15:23:08) Copyright (c) 1997-2014 The PHP Group Zend Engine v2.5.0, Copyright (c) 1998-2014 Zend Technologies with Zend OPcache v7.0.3, Copyright (c) 1999-2014, by Zend Technologies
PHP 5.5.22 (cli) (built: Feb 26 2015 16:05:53) Copyright (c) 1997-2015 The PHP Group Zend Engine v2.5.0, Copyright (c) 1998-2015 Zend Technologies with Zend OPcache v7.0.4-dev, Copyright (c) 1999-2015, by Zend Technologies

Источник

Curl cheatsheet

This Curl cheat sheet contains commands and examples of some common Curl tricks.

#Getting Started

#Introduction

Curl is a tool for transferring data between servers, supporting protocols, including:

  • HTTP
  • HTTPS
  • FTP
  • IMAP
  • LDAP
  • POP3
  • SCP
  • SFTP
  • SMB
  • SMTP
  • etc.
  • Curl GitHub source repository(github.com)
  • Curl Official Website(curl.se)

#Options

-o # --output: write to file -u user:pass # --user: authentication 
-v # --verbose: Make curl verbose during operation -vv # more verbose -s # --silent: don't show progress meter or errors -S # --show-error: When used with --silent (-sS), show errors but no progress meter 
-i # --include: include HTTP headers in the output -I # --head: header only

#Request

-X POST # --request -L # If the page redirects, follow the link -F # --form: HTTP POST data for multipart/form-data

#data

# --data: HTTP post data # URL encoding (eg, status="Hello") -d 'data' # --data pass file -d @file # --get: send -d data via get -G

#Header information Headers

-A # --user-agent -b name=val # --cookie -b FILE # --cookie -H "X-Foo: y" # --header --compressed # use deflate/gzip

#SSL

-E, --cert # --cert: client certificate file --cert-type # der/pem/eng -k, --insecure # For self-signed certificates

#Install

apk add --update curl # install in alpine linux

#Example

#CURL GET/HEAD

command description
curl -I https://quickref.me curl sends a request
curl -v -I https://quickref.me curl request with details
curl -X GET https://quickref.me use explicit http method for curl
curl —noproxy 127.0.0.1 http://www.stackoverflow.com curl without http proxy
curl —connect-timeout 10 -I -k https://quickref.me curl has no timeout by default
curl —verbose —header «Host: www.mytest.com:8182» quickref.me curl get extra header
curl -k -v https://www.google.com curl get response with headers

#Multiple file upload

#Prettify json output for curl response

$ curl -XGET http://${elasticsearch_ip}:9200/_cluster/nodes | python -m json.tool

#CURL POST

command description
curl -d «name=username&password=123456»

curl send request
curl -H «content-type: application/json» -d «{ \»woof\»: \»bark\»}» curl sends json

#CURL script install rvm

curl -sSL https://get.rvm.io | bash

#CURL Advanced

command description
curl -L -s http://ipecho.net/plain, curl -L -s http://whatismijnip.nl get my public IP
curl -u $username:$password http://repo.dennyzhang.com/README.txt curl with credentials
curl -v -F key1=value1 -F [email protected]

curl upload
curl -k -v —http2 https://www.google.com/ use http2 curl
curl -T cryptopp552.zip -u test:test ftp://10.32.99.187/ curl ftp upload
curl -u test:test ftp://10.32.99.187/cryptopp552.zip -o cryptopp552.zip curl ftp download
curl -v -u admin:admin123 —upload-file package1.zip http://mysever:8081/dir/package1.zip upload with credentials curl

#Check website response time

curl -s -w \ '\nLookup time:\t%{time_namelookup}\nConnect time:\t%{time_connect}\nAppCon time:\t%{time_appconnect}\nRedirect time:\t%{time_redirect}\nPreXfer time:\t%{time_pretransfer }\nStartXfer time:\t%{time_starttransfer}\n\nTotal time:\t%{time_total}\n' \ -o /dev/null https://www.google.com

#Use Curl to check if a remote resource is available

curl -o /dev/null --silent -Iw "%{http_code}" https://example.com/my.remote.tarball.gz

#Downloading file

curl https://example.com | \ grep --only-matching 'src="https://quickref.me/[%5E"]*.[png]"' | \ cut -d \" -f2 | \ while read i; do curl https://example.com/"${i}" \ -o "${i##*/}"; done

Download all PNG files from the site (using GNU grep)

#Download the file, save the file without changing its name

curl --remote-name "https://example.com/linux-distro.iso" 
curl --remote-name "http://example.com/index.html" --output foo.html

#continue partial download

curl --remote-name --continue-at -"https://example.com/linux-distro.iso"

#Download files from multiple domains

curl "https://www.{example,w3,iana}.org/index.html" --output "file_#1.html"

#Download a series of files

curl "https://{foo,bar}.com/file_2.webp" --output "#1_#2.webp"

Download a series of files (output foo_file1.webp , foo_file2.webp. bar_file1_webp , etc.)

#Redirect output to file

#Basic Authentication

$ curl --user username:password http://example.com/ $ curl -u username:password http://example.com/

#Write to file instead of stdout

$ curl -o file http://url/file $ curl --output file http://url/file

#Download header information

$ curl -I url # display header information

#Write output to a file named remote_file

$ curl -o file http://url/file $ curl --output file http://url/file

#Execute remote script

#Configuration file

curl -K file # read configuration from file curl --config file $HOME/.curlrc # default configuration file on UNIX-like systems

Источник

Читайте также:  Json parsing using python
Оцените статью
© 2023 Программирование