Use Case:- I am working on a web application which allows to create HTML templates and publish them on amazon S3.Now to publish the websites I use nginx as a proxy server.
What the proxy server does is,when a user enters the website URL,I want to identify how to check if the request comes from my application i.e app.mysite.com(This won't change) and route it to apache for regular access,if its coming from some other domain like a regular URL www.mysite.com(This needs to be handled dynamically.Can be random) it goes to the S3 bucket that hosts the template.
My current configuration is:
user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
charset utf-8;
keepalive_timeout 65;
server_tokens off;
sendfile on;
tcp_nopush on;
tcp_nodelay off;
Default Server Block to catch undefined host names
server {
listen 80;
server_name app.mysite.com;
access_log off;
error_log off;
location / {
proxy_pass http://127.0.0.1:8080;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_redirect off;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_connect_timeout 90;
proxy_send_timeout 90;
proxy_read_timeout 90;
client_max_body_size 10m;
client_body_buffer_size 128k;
proxy_buffer_size 4k;
proxy_buffers 4 32k;
proxy_busy_buffers_size 64k;
}
}
}
Load all the sites
include /etc/nginx/conf.d/*.conf;
Updates as I was not clear enough :-
My question is how can I handle both the domains in the config file.My nginx is a proxy server on port 80 on an EC2 instance.This also hosts my application that runs on apache on a differnet port.So any request coming for my application will come from a domain app.mysite.com and I also want to proxy the hosted templates on S3 which are inside a bucket say sites.mysite.com/coolsite.com/index.html.So if someone hits coolsite.com I want to proxy it to the folder sites.mysite.com/coolsite.com/index.html and not to app.syartee.com.Hope I am clear
The other server block:
# Server for S3
server {
# Listen on port 80 for all IPs associated with your machine
listen 80;
# Catch all other server names
server_name _; //I want it to handle other domains then app.mysite.com
# This code gets the host without www. in front and places it inside
# the $host_without_www variable
# If someone requests www.coolsite.com, then $host_without_www will have the value coolsite.com
set $host_without_www $host;
if ($host ~* www\.(.*)) {
set $host_without_www $1;
}
location / {
# This code rewrites the original request, and adds the host without www in front
# E.g. if someone requests
# /directory/file.ext?param=value
# from the coolsite.com site the request is rewritten to
# /coolsite.com/directory/file.ext?param=value
set $foo 'http://sites.mysite.com';
# echo "$foo";
rewrite ^(.*)$ $foo/$host_without_www$1 break;
# The rewritten request is passed to S3
proxy_pass http://sites.mysite.com;
include /etc/nginx/proxy_params;
}
}
Also I understand I will have to make the DNS changes in the cname of the domain.I guess I will have to add app.mysite.com under the CNAME of the template domain name?Please correct if wrong.
Thank you for your time
Found this as a part of documentation but took a while to understand.
I had to add a default_server attribute in the second server block for rest of the domains to work.
If we config the server_name, nginx will serve the content based on config block that match the domain.
app.mysite.com
server {
listen 80;
server_name app.mysite.com;
# config for app.mysite.com
}
other websites
server {
listen 80 default_server; #To handle domains apart from the fixed domain(In my case app.mysite.com)
server_name _ ;
# config for coolsite.com, anotherdomain.com ...
}
I think you need to remove $foo from
rewrite ^(.*)$ $foo/$host_without_www$1 break;
With the foo you would pass
http://sites.mysite.com/coolsite.com/directory/file.ext?param=value
to
proxy_pass http://sites.mysite.com;
Just a guess
Related
So I am running an Express.js/Passport.js web server with Nginx on CentOS 7.
Here's the problem: I cannot display any endpoint that starts with w. Trying to do so results in a Cannot GET /wiki/Main_Page message. However, changing that same route from e.g. /welcome to /selcome works just fine.
Most likely culprit:
I installed Mediawiki but removed it not long after. IIRC there was some kind of setting that prettied up URLs that starting with /w after the domain. So I'm guessing a rewrite rule is persisting... just have no idea where.
Here are my config files:
/srv/node/example.com/app/routes/auth.js
module.exports = function(app, passport) {
//Changing the URLs below from welcome to selcome works fine
app.get('/activate', passport.authenticate('user-activate-account', {
successRedirect: '/welcome',
failureRedirect: '/404',
failureFlash: true
}));
...
app.get('/welcome', isLoggedIn, function(req, res) {
res.render('inside', {
page_title: 'Welcome!',
inc_style: true,
style_sheet: 'style/dashboard.css',
portal: function() {
return 'welcome';
}
});
});
};
/etc/nginx.conf
user nginx nginx;
error_log /var/log/nginx/error.log info; # [ debug | info | notice | warn | error | crit ]
events {
worker_connections 1024;
}
http {
include mime.types;
include /etc/nginx/sites_enabled/.conf; //Really *.conf but more readable this way
include /etc/letsencrypt/options-ssl-nginx.conf;
server_names_hash_bucket_size 64;
# Compression - requires gzip and gzip static modules.
gzip on;
gzip_static on;
gzip_vary on;
gzip_http_version 1.1;
gzip_min_length 700;
# Compression levels over 6 do not give an appreciable improvement
# in compression ratio, but take more resources.
gzip_comp_level 6;
# IE 6 and lower do not support gzip with Vary correctly.
gzip_disable "msie6";
# Before nginx 0.7.63:
#gzip_disable "MSIE [1-6]\.";
# Redirect http traffic to https
server {
listen 80 default_server;
server_name _;
return 301 https://$host$request_uri;
}
# Catch-all server for requests to invalid hosts.
# Also catches vulnerability scanners probing IP addresses.
server {
listen 443 ssl;
server_name bogus;
root /var/empty;
return 444;
location / {
try_files $uri $uri/ =404;
}
}
# If running php as fastcgi, specify php upstream.
upstream php {
server unix:/var/run/php7.2-fpm.socket;
}
}
/etc/nginx/sites_available/example.com.conf
#sub.example.com
server {
ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem; # managed by Certbot
listen 443 ssl;
server_name sub.example.com;
access_log /var/log/nginx/sub.example.com.access.log;
error_log /var/log/nginx/sub.example.com.error.log;
location /something\.js {
alias /var/www/html/example.com/sub.example.com/design/;
location ~* \.(gif|jpe?g|png|svg|webm)$ {
try_files $uri =404;
expires 30d;
}
}
location / {
proxy_pass http://localhost:3847;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
}
}
So far I have
Purged Nginx config files, and every config file in the include tree, of any legacy rules.
Cleared Nginx cache
My node.js project folder does not even contain the string wiki anywhere except for NPM links in the node_modules folder.
All the PHP files in /etc, /etc/php.d, /etc/php-fpm.d do not contain the string wiki
Deleted mediawiki folder
Restarted nginx
Restarted php-fpm
Restarted entire machine
I'm genuinely baffled at where this problem could be. Any ideas?
Turns out it was the browser. Despite clearing history multiple times and using incognito mode, nothing changed. But using another browser worked flawlessly. I feel like an idiot, but at least I can finally put this rage-inducing mystery behind me.
Was using: Firefox (I guess private mode isn't enough)
Chrome showed it was a browser error.
I have a task that has two containers on fargate: one java and one php/nginx container.
The java container maps its internal port 80 to the extenral port 8080 and the php container maps its internal port 80 to the external port 8081.
The task is using the awsvpc network mode. According to the aws documentation, they should be able to communicate with each other via localhost once they are on fargate.
I tried testing this by having the Java container send a request to the PHP container using "localhost:8081":
public boolean isPHPReachable() {
System.out.println("Attempting to make request to PHP");
try {
final URL url = new URL("http://localhost:8081/test");
final HttpURLConnection con = (HttpURLConnection) url.openConnection();
con.setRequestMethod("GET");
final int status = con.getResponseCode();
System.out.printf("Get request to PHP resulted in a %d: %s", status, con.getResponseMessage());
return status == 200;
} catch (final IOException err) {
err.printStackTrace();
return false;
}
The php code it should be hitting is this:
<?php
if ($_GET['q'] === 'test') {
echo('healthy');
die;
}
The 'q' is set by nginx (see below config).
However,I see that the cloudwatch logs are showing an eror when Java tries to send PHP the request:
22:55:04
Attempting to make request to PHP
22:55:04
java.net.ConnectException: Connection refused (Connection refused)
I tried to contact the PHP container directly, but I get timedout. This does not occur when I attempt to reach Java.
Potential Problem 1: Nginx is misconfigured
I suspect this might be an nginx issue. However, I am not able to reproduce the problem when I create the PHP container on my local machine -- it's reachable when I ping localhost:8081.
The problem is only occuring in Fargate.
Here is my configuration:
# For more information on configuration, see:
# * Official English Documentation: http://nginx.org/en/docs/
# * Official Russian Documentation: http://nginx.org/ru/docs/
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;
# Load dynamic modules. See /usr/share/doc/nginx/README.dynamic.
include /usr/share/nginx/modules/*.conf;
events {
worker_connections 1024;
}
http {
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
include /etc/nginx/mime.types;
default_type application/octet-stream;
# Load modular configuration files from the /etc/nginx/conf.d directory.
# See http://nginx.org/en/docs/ngx_core_module.html#include
# for more information.
include /etc/nginx/conf.d/*.conf;
server {
# Reject requests not matching any server block
listen 80 default_server;
server_name "";
return 410;
}
server {
listen 80;
root /var/www/public_html;
server_name localhost;
index index.php;
set_real_ip_from 0.0.0.0/0; # we trust that the webserver is firewalled
real_ip_header X-Forwarded-For;
set $webroot $document_root;
error_page 404 410 /404_empty.html;
autoindex off;
# Disable server signature
# (i.e. remove "nginx 1.10" or similar from error pages and response headers)
server_tokens off;
#################
# PHP locations #
#################
location / {
rewrite ^/(.*)$ /sites/all/index.php?q=$1;
error_page 404 /sites/all/index.php;
}
}
2. Misconfigured security groups:
I do not believe this is the case. The security group attached to this container allows inbound TCP requests to any port from any destination. It also allows all outbound requests.
Anyone spot an immediate issue? Any debugging tips?
I've just installed a Ghost Blog on a new server running NGINX. The Ghost config.json file is pointing at the correct directory /blog and the blog loads fine when I visit it.
What isn't working is when I remove /blog from the URL, I get taken to a 404 page. I've checked my sites-enabled file, which looks like this:
server {
listen 80;
listen [::]:80;
server_name *********;
root /var/www/ghost/system/nginx-root;
location ^~ /blog {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $http_host;
proxy_pass http://*********:2368;
proxy_redirect off;
}
location ~ /.well-known {
allow all;
}
client_max_body_size 50m;
But I'm not entirely sure what I need to change to not get the 404 error. I have an example .php file which should be loading but isn't.
I've always used the Digital Ocean One-Click Ghost app but I wanted to use the Ghost CLI this time round. I have a feeling I've missed something though.
the following may remove some of your restrictions but it will work
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name _;
return 301 https://$host$request_uri;
}
server {
listen 443 ssl default_server;
listen [::]:443 ssl default_server;
server_name _;
ssl on;
ssl_certificate /etc/letsencrypt/live/thedomain.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/thedomain.com/privkey.pem;
access_log /var/log/nginx/thedomain.access.log;
error_log /var/log/nginx/thedomain.error.log;
root /var/www/thedomain;
index index.html;
gzip on;
gzip_proxied any;
gzip_types text/css text/javascript text/xml text/plain application/javascript application/x-javascript application/json;
location / {
try_files $uri $uri/ =404;
}
}
You need to make sure all the ssl files are there and permissioned for access by www-data.
If you need to run certbot for the first time, just but the 443 code in an 80 block without the ssl statements
The nginx configuration you've posted only deals with Ghost.
You've setup a server responding on port 80, set the root to Ghost's nginx-root, and created 2 location blocks. One is for /blog/ and serves Ghost, the second .well-known block is for handling generation of SSL certificates with letsencrypt.
I'm not an expert at configuring nginx for PHP, but this guide from Digital Ocean and this stackoverflow question covers a lot of the details.
I think you have a couple of options:
Set the index to be index.php
Add a new location block for / which serves php files
Add a block to handle all php files
I believe adding a new location block like this, will mean any .php files you have will always be called if the path in the URL matches.
location ~ \.php$ {
try_files $uri =404;
fastcgi_split_path_info ^(.+\.php)(/.+)$;
fastcgi_pass unix:/var/run/php5-fpm.sock;
fastcgi_index index.php;
include fastcgi_params;
}
But the value of fastcgi_pass will depend on how you have PHP setup on your system.
With this, going to /index.php should work.
Setting index: index.php will mean that / maps to /index.php I'm not sure if that will interfere with Ghost, if it does, you'd need a specific location / {} block instead of the index being set.
I have a challenge that I can't figure out how to solve.
I a DigitalOcean VPS running CentOS7. Here I host a domain, e.q. www.example.com.
I'm running nginx right now on it. It used to be Apache but I could not figure it out how to make the websockets reversed proxy, for meteor work with Apache.
Now in Nginx I created a vhost configuration to load for the main domain, www.example.com and example.com, from a meteor server that runs with pm2-meteor:
server_tokens off; # for security-by-obscurity: stop displaying nginx version
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
server {
listen 443 ssl spdy; # we enable SPDY here
server_name www.example.com; # this domain must match Common Name (CN) in the SSL certificate
root /var/www/html/example/bundle; # irrelevant
index index.html; # irrelevant
ssl_certificate /etc/httpd/ssl/example/example.crt; # full path to SSL certificate and CA certificate concatenated together
ssl_certificate_key /etc/httpd/ssl/example/example.key; # full path to SSL key
# performance enhancement for SSL
ssl_stapling off;
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 5m;
# safety enhancement to SSL: make sure we actually use a safe cipher
ssl_prefer_server_ciphers on;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:ECDHE-RSA-RC4-SHA:ECDHE-ECDSA-RC4-SHA:RC4-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!3DES:!MD5:!PSK';
# config to enable HSTS(HTTP Strict Transport Security) https://developer.mozilla.org/en-US/docs/Security/HTTP_Strict_Transport_Security
# to avoid ssl stripping https://en.wikipedia.org/wiki/SSL_stripping#SSL_stripping
add_header Strict-Transport-Security "max-age=31536000;";
# If your application is not compatible with IE <= 10, this will redirect visitors to a page advising a browser update
# This works because IE 11 does not present itself as MSIE anymore
#if ($http_user_agent ~ "MSIE" ) {
# return 303 https://browser-update.org/update.html;
#}
# pass all requests to Meteor
location / {
proxy_pass http://127.0.0.1:4001;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade; # allow websockets
proxy_set_header Connection $connection_upgrade;
proxy_set_header X-Forwarded-For $remote_addr; # preserve client IP
# this setting allows the browser to cache the application in a way compatible with Meteor
45,1 Top
I'm running the meteor server on 4001 port.
Ok, this works perfectly for meteor. My problem now is that I want to create a vhost for a subdomain, payment.example.com, which will load larval 5.
Normally I want this subdomain to load over HTTPS also.
How on earth can I do this without loading the default nginx vhost settings or the domain vhost which loads meteor?
I can't figure out a way to make it work. This is the payment subdomain virtual host I'm trying to make and it's not working:
server {
listen 431 ssl spdy; # we enable SPDY here
listen [::]:431 ipv6only=on default_server;
server_name payment.example.ro; # this domain must match Common Name (CN) in the SSL certificate
root /var/www/html/example/payment/;
access_log /var/log/nginx/nginx_access.log;
error_log /var/log/nginx/nginx_error.log;
ssl_certificate /etc/httpd/ssl/example/example.crt; # full path to SSL certificate and CA certificate concatenated together
ssl_certificate_key /etc/httpd/ssl/example/example.key; # full path to SSL key
# performance enhancement for SSL
ssl_stapling off;
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 5m;
# safety enhancement to SSL: make sure we actually use a safe cipher
ssl_prefer_server_ciphers on;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:ECDHE-RSA-RC4-SHA:ECDHE-ECDSA-RC4-SHA:RC4-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!3DES:!MD5:!PSK';
# config to enable HSTS(HTTP Strict Transport Security) https://developer.mozilla.org/en-US/docs/Security/HTTP_Strict_Transport_Security
# to avoid ssl stripping https://en.wikipedia.org/wiki/SSL_stripping#SSL_stripping
add_header Strict-Transport-Security "max-age=31536000;";
location / {
root /var/www/html/example/payment/;
index index.php index.html index.htm;
}
location ~ \.php$ {
try_files $uri =404;
fastcgi_pass unix:/var/run/php-fpm/php-fpm.sock;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
include fastcgi_params;
}
}
Now when I load payment.example.com I load the meteor reversed proxy.
Any suggestions or ideas on what to do to make this work. Is this even possible?
Do I have to put it in a subfolder rather that subdomain? I prefer the subdomain...
Thank you guy!
I am using WT-NMP software with combination of php,mysql and ngnix server.
worker_processes 1;
events {
worker_connections 1024;
}
http {
server_tokens off;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
ssi off;
#Timeouts
client_body_timeout 5;
client_header_timeout 5;
keepalive_timeout 25 25;
send_timeout 15s;
resolver_timeout 3s;
#Directive sets timeout period for connection with FastCGI-server. It should be noted that this value can't exceed 75 seconds.
fastcgi_connect_timeout 5s;
#Directive sets the amount of time for upstream to wait for a fastcgi process to send data. Change this directive if you have long running fastcgi processes that do not produce output until they have finished processing. If you are seeing an upstream timed out error in the error log, then increase this parameter to something more appropriate.
fastcgi_read_timeout 40s;
#Directive specifies request timeout to the server. The timeout is calculated between two write operations, not for the whole request. If no data have been written during this period then serve closes the connection.
fastcgi_send_timeout 15s;
fastcgi_buffers 8 32k;
fastcgi_buffer_size 32k;
#fastcgi_busy_buffers_size 256k;
#fastcgi_temp_file_write_size 256k;
open_file_cache off;
#php max upload limit cannot be larger than this
client_max_body_size 8m;
####client_body_buffer_size 1K;
client_header_buffer_size 1k;
large_client_header_buffers 2 1k;
types_hash_max_size 2048;
include nginx.mimetypes.conf;
default_type text/html;
##
# Logging Settings
##
access_log "c:/wt-nmp/log/nginx_access.log";
error_log "c:/wt-nmp/log/nginx_error.log" warn; #debug or warn
log_not_found on; #enables or disables messages in error_log about files not found on disk.
rewrite_log off;
#Leave this off
fastcgi_intercept_errors off;
gzip off;
index index.php index.htm index.html;
server {
listen 127.0.0.1:80 default_server;
listen 127.0.0.1:8080;
#listen [::1]:80 ipv6only=on;
server_name mylocalhost;
root "c:/wt-nmp/www/projectname";
autoindex on;
error_log "c:/wt-nmp/log/nginx_error.log";
allow 127.0.0.1;
#allow ::1;
deny all;
# deny access to .htaccess files, if Apache's document root
# concurs with nginx's one
#
location ~ /\.ht {
deny all;
}
#tools are now served from wt-nmp/include/tools/
location ~ ^/tools/.*\.php$ {
root "c:/wt-nmp/include";
try_files $uri =404;
include nginx.fastcgi.conf;
fastcgi_pass php_farm;
}
location ~ ^/tools/ {
root "c:/wt-nmp/include";
}
location ~ \.php$ {
try_files $uri =404;
fastcgi_pass php_farm;
include nginx.fastcgi.conf;
}
}
include domains.d/*.conf;
include nginx.phpfarm.conf;
}
when I am trying to access with "mylocalhost" its working fine when I am firing an event and call ajax method . It is giving page not found message
WT-NMP - portable Nginx Mysql Php development stack for Windows README.md states:
Starting only one PHP-CGI server with wt-nmp.exe --phpCgiServers=1 will result in slow ajax requests since Nginx will not be able to process PHP scripts simultaneous.
So, make sure you use the latest version of WT-NMP and choose at least 3 PHP-CGI servers.