cleanup
This commit is contained in:
@ -1,8 +0,0 @@
|
||||
root = true
|
||||
|
||||
[*]
|
||||
indent_style = space
|
||||
indent_size = 4
|
||||
trim_trailing_whitespace = true
|
||||
end_of_line = lf
|
||||
insert_final_newline = true
|
||||
2
.gitignore
vendored
2
.gitignore
vendored
@ -1,2 +0,0 @@
|
||||
scripts/output
|
||||
scripts/config.json
|
||||
21
LICENSE
21
LICENSE
@ -1,21 +0,0 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2017 UK LAN Techs
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
106
README.md
106
README.md
@ -1,106 +0,0 @@
|
||||
# Game Content Cache Hostnames
|
||||
|
||||
## Introduction
|
||||
|
||||
This is a list of all hostnames that are required to be redirected for running a game content cache. This list will change frequently so this is designed to be a definitive list.
|
||||
|
||||
## Usage
|
||||
|
||||
You can use this list one of two ways:
|
||||
|
||||
- Overriding DNS for these hostnames to point to the IP of your cache server.
|
||||
- Use them in Squid with WCCP to redirect content to the right cache server.
|
||||
|
||||
There is a cache_domains.json file to define CDNs and additional metadata with the following structure
|
||||
|
||||
- cache_domains: Array of cache_domain object
|
||||
- name: shortname for the cache domain. Should match `^[0-9A-Za-z]$`
|
||||
- description: a longer description to aid others in identifying what this domain does (not all users of this repo will want to enable all caches)
|
||||
- notes: implementation specific notes which may be useful for other users
|
||||
- domain_files: array of files within the repo associated to the cdn. Most CDNs only need one file
|
||||
- Example domain entry for steam
|
||||
```json
|
||||
{
|
||||
"cache_domains": [
|
||||
{
|
||||
"name": "steam",
|
||||
"description": "CDN for steam platform",
|
||||
"domain_files": ["steam.txt"]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Host file format
|
||||
|
||||
There is a separate file for each cacheable service. Some notes on formatting:
|
||||
|
||||
- Every line should be a separate hostname for that service.
|
||||
- Only one entry is permitted per line.
|
||||
- Wildcards are permitted as per below
|
||||
- Lines starting with a # will be treated as a comment.
|
||||
- Files must end with an empty newline.
|
||||
- File names must match the regular expression `^[0-9A-Za-z].txt`
|
||||
|
||||
#### Wildcards
|
||||
|
||||
The wildcard format shall be defined as per the below
|
||||
|
||||
- Wildcards should be represented with an asterisk.
|
||||
- If a wildcard is used, it should be the first character on the line.
|
||||
- Wildcards are not treated as matching null, e.g. `*.example.com` will match `a.example.com` but will not match `example.com`
|
||||
- Only simple domain wildcards will be accepted e.g. `*.example.com` not `*ww.example.com`
|
||||
|
||||
##### Notes for Squid users
|
||||
|
||||
If you are using these files within a squid dst_domain acl you will need to reformat the wildcard entries to be compliant with the squid acl format. The following regex should suffice `s/*\./\./` however latest versions of squid are very particular about duplicate domains. The scripts/create-squid.sh should help you out here!
|
||||
|
||||
## Updates
|
||||
|
||||
Please fork this repository and submit pull requests if you have any extra hostnames or services to add. We want this list to be definitive and collaborative!
|
||||
|
||||
### New CDNs
|
||||
|
||||
If the CDN you are adding does not already exist please remember to add an entry within cache_domains.json as well as a new .txt file
|
||||
|
||||
### Uncacheable CDNs
|
||||
|
||||
Certain CDNs are not suitable for inclusion on this list because the content is entirely delivered over HTTPS and thus cannot be cached.
|
||||
|
||||
For a list of currently identified uncacheable CDNs please see these issues: https://github.com/uklans/cache-domains/issues?q=is%3Aissue+is%3Aopen+label%3Ahttps-cantfix
|
||||
|
||||
## Frequently Asked Questions
|
||||
|
||||
If you have any questions, please check [our FAQs](faq.md).
|
||||
|
||||
## Issues and Feedback
|
||||
|
||||
Please raise all issues and feedback on GitHub at [uklans/cache-domains](https://github.com/uklans/cache-domains/issues).
|
||||
|
||||
## Useful Information
|
||||
|
||||
There are a few useful anecdotes on cache domains usage in some old issues https://github.com/uklans/cache-domains/issues?q=is%3Aissue+is%3Aclosed+label%3Afor-reference-only
|
||||
|
||||
## License
|
||||
|
||||
The MIT License (MIT)
|
||||
|
||||
Copyright (c) 2017 UK LAN Techs
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@ -1,3 +0,0 @@
|
||||
assetcdn.101.arenanetworks.com
|
||||
assetcdn.102.arenanetworks.com
|
||||
assetcdn.103.arenanetworks.com
|
||||
11
blizzard.txt
11
blizzard.txt
@ -1,11 +0,0 @@
|
||||
dist.blizzard.com
|
||||
dist.blizzard.com.edgesuite.net
|
||||
blizzard.vo.llnwd.net
|
||||
blzddist1-a.akamaihd.net
|
||||
blzddist2-a.akamaihd.net
|
||||
blzddist3-a.akamaihd.net
|
||||
level3.blizzard.com
|
||||
nydus.battle.net
|
||||
edge.blizzard.top.comcast.net
|
||||
cdn.blizzard.com
|
||||
*.cdn.blizzard.com
|
||||
@ -1,131 +0,0 @@
|
||||
{
|
||||
"cache_domains": [
|
||||
{
|
||||
"name": "arenanet",
|
||||
"description": "CDN for guild wars, HoT",
|
||||
"domain_files": ["arenanet.txt"]
|
||||
},
|
||||
{
|
||||
"name": "blizzard",
|
||||
"description": "CDN for blizzard/battle.net",
|
||||
"domain_files": ["blizzard.txt"]
|
||||
},
|
||||
{
|
||||
"name": "cod",
|
||||
"description": "CDN for Call of Duty",
|
||||
"domain_files": ["cod.txt"]
|
||||
},
|
||||
{
|
||||
"name": "bsg",
|
||||
"description": "CDN for Battle State Games, Tarkov",
|
||||
"domain_files": ["bsg.txt"]
|
||||
},
|
||||
{
|
||||
"name": "cityofheroes",
|
||||
"description": "CDN for City of Heroes (Homecoming)",
|
||||
"domain_files": ["cityofheroes.txt"]
|
||||
},
|
||||
{
|
||||
"name": "daybreak",
|
||||
"description": "Daybreak games CDN",
|
||||
"domain_files": ["daybreak.txt"]
|
||||
},
|
||||
{
|
||||
"name": "epicgames",
|
||||
"description": "CDN for Epic Games",
|
||||
"domain_files": ["epicgames.txt"]
|
||||
},
|
||||
{
|
||||
"name": "frontier",
|
||||
"description": "CDN for frontier games",
|
||||
"domain_files": ["frontier.txt"]
|
||||
},
|
||||
{
|
||||
"name": "nexusmods",
|
||||
"description": "Nexus mods / skyrim content",
|
||||
"domain_files": ["nexusmods.txt"]
|
||||
},
|
||||
{
|
||||
"name": "neverwinter",
|
||||
"description": "Cryptic CDN for Neverwinter",
|
||||
"domain_files": ["neverwinter.txt"]
|
||||
},
|
||||
{
|
||||
"name": "nintendo",
|
||||
"description": "CDN for Nintendo consoles and download servers",
|
||||
"domain_files": ["nintendo.txt"]
|
||||
},
|
||||
{
|
||||
"name": "origin",
|
||||
"description": "CDN for origin",
|
||||
"notes": "Should be enabled for HTTP traffic only or with a HTTPS proxy else origin client download fails",
|
||||
"mixed_content": true,
|
||||
"domain_files": ["origin.txt"]
|
||||
},
|
||||
{
|
||||
"name": "pathofexile",
|
||||
"description": "CDN for Path Of Exile",
|
||||
"domain_files": ["pathofexile.txt"]
|
||||
},
|
||||
{
|
||||
"name": "renegadex",
|
||||
"description": "CDN for Renegade X",
|
||||
"domain_files": ["renegadex.txt"]
|
||||
},
|
||||
{
|
||||
"name": "riot",
|
||||
"description": "CDN for riot games",
|
||||
"domain_files": ["riot.txt"]
|
||||
},
|
||||
{
|
||||
"name": "rockstar",
|
||||
"description": "CDN for rockstar games",
|
||||
"domain_files": ["rockstar.txt"]
|
||||
},
|
||||
{
|
||||
"name": "sony",
|
||||
"description": "CDN for sony / playstation",
|
||||
"domain_files": ["sony.txt"]
|
||||
},
|
||||
{
|
||||
"name": "square",
|
||||
"description": "CDN for Final Fantasy XIV",
|
||||
"domain_files": ["square.txt"]
|
||||
},
|
||||
{
|
||||
"name": "steam",
|
||||
"description": "CDN for steam platform",
|
||||
"domain_files": ["steam.txt"]
|
||||
},
|
||||
{
|
||||
"name": "uplay",
|
||||
"description": "CDN for uplay downloader",
|
||||
"domain_files": ["uplay.txt"]
|
||||
},
|
||||
{
|
||||
"name": "teso",
|
||||
"description": "CDN for The Elder Scrolls Online",
|
||||
"domain_files": ["teso.txt"]
|
||||
},
|
||||
{
|
||||
"name": "warframe",
|
||||
"description": "CDN for Warframe",
|
||||
"domain_files": ["warframe.txt"]
|
||||
},
|
||||
{
|
||||
"name": "wargaming",
|
||||
"description": "CDN for wargaming.net",
|
||||
"domain_files": ["wargaming.net.txt"]
|
||||
},
|
||||
{
|
||||
"name": "wsus",
|
||||
"description": "CDN for windows updates",
|
||||
"domain_files": ["windowsupdates.txt"]
|
||||
},
|
||||
{
|
||||
"name": "xboxlive",
|
||||
"description": "CDN for xboxlive",
|
||||
"domain_files": ["xboxlive.txt"]
|
||||
}
|
||||
]
|
||||
}
|
||||
@ -1,4 +0,0 @@
|
||||
cdn-na1.homecomingservers.com
|
||||
cdn-na2.homecomingservers.com
|
||||
cdn-na3.homecomingservers.com
|
||||
cdn-eu1.homecomingservers.com
|
||||
2
cod.txt
2
cod.txt
@ -1,2 +0,0 @@
|
||||
cod-assets.cdn.callofduty.com
|
||||
cod-assets.cdn.callofduty.com.edgesuite.net
|
||||
@ -1,6 +0,0 @@
|
||||
# Daybreak games
|
||||
#
|
||||
# Do NOT cache manifest.patch.daybreakgames.com
|
||||
#
|
||||
#PS2
|
||||
pls.patch.daybreakgames.com
|
||||
@ -1,13 +0,0 @@
|
||||
cdn1.epicgames.com
|
||||
cdn2.epicgames.com
|
||||
cdn.unrealengine.com
|
||||
cdn1.unrealengine.com
|
||||
cdn2.unrealengine.com
|
||||
cdn3.unrealengine.com
|
||||
download.epicgames.com
|
||||
download2.epicgames.com
|
||||
download3.epicgames.com
|
||||
download4.epicgames.com
|
||||
epicgames-download1.akamaized.net
|
||||
fastly-download.epicgames.com
|
||||
cloudflare.epicgamescdn.com
|
||||
41
faq.md
41
faq.md
@ -1,41 +0,0 @@
|
||||
# Frequently Asked Questions
|
||||
|
||||
## What is this list?
|
||||
|
||||
This is a list of hostnames for various CDNs or companies that host game related content that can be cached.
|
||||
|
||||
## Who is this list for?
|
||||
|
||||
The list is maintained primarily for people running LAN parties / gaming events, whether that's 10 people gaming in a friend's garage or thousands bringing their own machines to an exhibition centre. Anyone running a LAN will likely benefit from caching game downloads and other suitable content, by saving internet bandwidth and preventing multiple people downloading the same content from the internet. However anyone running any network that includes multiple gaming computers (such as an internet or gaming cafe) may also benefit.
|
||||
|
||||
## How do I use this list?
|
||||
|
||||
There are multiple ways to use this list in a caching solution. See [the readme](https://github.com/uklans/cache-domains#usage) for more information.
|
||||
|
||||
## Do you recommend any cache solutions?
|
||||
|
||||
Many of the maintainers of this repo also contribute to [the lancachenet project](https://github.com/lancachenet/monolithic) which uses this list as its source of host names.
|
||||
|
||||
## Why isn't *some other service* on the list?
|
||||
|
||||
There are several reasons why a particular service / CDN / website might not be on this list. Here are some of the more common ones:
|
||||
|
||||
1. It's not technically possible to cache it. Many popular websites, including video streaming sites and even some games CDN's use SSL Encryption (i.e https URLs) to serve their content. Because the client opens a secure connection directly to the host, there is no way for the network operator to see what they are downloading, nor cache it. Whilst there are several approaches to work around this, such as MITM techniques, they usually rely on control over the client device to affect SSL Certificates - control somebody running a BYOC LAN typically does not have over the devices customers bring.
|
||||
|
||||
- [These issues](https://github.com/uklans/cache-domains/issues?q=is%3Aissue+is%3Aopen+label%3Ahttps-cantfix) contain game CDNs that we would like to include, but cannot for this reason.
|
||||
|
||||
2. It's out of scope for a LAN. We try to keep this list targeted towards people running LANs. Whilst some none game-related CDNs are included for things like windows updates that use internet bandwidth at LANs, we do not go searching for unrelated sites / hostnames.
|
||||
|
||||
3. It's not a good cache target / it would not get a good hit ratio. Game downloads are a great cache target because they are large, remain the same for every user and are likely to be downloaded multiple times at a LAN. Other hostnames that only serve dynamic or media files, or content that is not likely to be downloaded multiple times is not a good cache target and can waste valuable storage space on your cache server. This can lead to potentially more valuable content being evicted from the cache due to low space.
|
||||
|
||||
4. We simply don't yet have a tested list of hostnames for it yet. This is the category you can help with - if you have something that doesn't fall into one of the above reasons not to include it, we would love to review your PR. See [the readme](https://github.com/uklans/cache-domains) for instructions on how to add a new CDN.
|
||||
|
||||
## SNI Proxy / HTTPS
|
||||
|
||||
[lancachenet/sniproxy](https://github.com/lancachenet/sniproxy) is part of the lancache project and allows hostnames that serve BOTH http and https content to be included in this list. Traffic going to that hostname on port 80 (http) will hit lancache and be cached, whilst traffic on port 443 (HTTPS) is passed straight through to the internet by sniproxy.
|
||||
|
||||
It does not allow https / SSL content to be inspected or cached. Hostnames that serve all or almost all https traffic are still unlikely to be good candidates for this list, as it just places load on the cache box but does not save any internet bandwidth.
|
||||
|
||||
## How can I test an addition to this list?
|
||||
|
||||
If you are using the lancachenet project, you can fork this repo, add your test hostnames and then use [these instructions](https://github.com/lancachenet/lancache-dns#custom-forksbranches) to use it with your cache instance rather than the main repo.
|
||||
@ -1 +0,0 @@
|
||||
cdn.zaonce.net
|
||||
@ -1 +0,0 @@
|
||||
level3.nwhttppatch.crypticstudios.com
|
||||
@ -1 +0,0 @@
|
||||
filedelivery.nexusmods.com
|
||||
12
nintendo.txt
12
nintendo.txt
@ -1,12 +0,0 @@
|
||||
ccs.cdn.wup.shop.nintendo.net
|
||||
ccs.cdn.wup.shop.nintendo.net.edgesuite.net
|
||||
geisha-wup.cdn.nintendo.net
|
||||
geisha-wup.cdn.nintendo.net.edgekey.net
|
||||
idbe-wup.cdn.nintendo.net
|
||||
idbe-wup.cdn.nintendo.net.edgekey.net
|
||||
ecs-lp1.hac.shop.nintendo.net
|
||||
receive-lp1.dg.srv.nintendo.net
|
||||
*.wup.shop.nintendo.net
|
||||
*.wup.eshop.nintendo.net
|
||||
*.hac.lp1.d4c.nintendo.net
|
||||
*.hac.lp1.eshop.nintendo.net
|
||||
@ -1,3 +0,0 @@
|
||||
origin-a.akamaihd.net
|
||||
lvlt.cdn.ea.com
|
||||
cdn-patch.swtor.com
|
||||
@ -1 +0,0 @@
|
||||
patchcdn.pathofexile.com
|
||||
@ -1,2 +0,0 @@
|
||||
patches.totemarts.services
|
||||
patches.totemarts.games
|
||||
5
riot.txt
5
riot.txt
@ -1,5 +0,0 @@
|
||||
l3cdn.riotgames.com
|
||||
worldwide.l3cdn.riotgames.com
|
||||
riotgamespatcher-a.akamaihd.net
|
||||
riotgamespatcher-a.akamaihd.net.edgesuite.net
|
||||
*.dyn.riotcdn.net
|
||||
@ -1 +0,0 @@
|
||||
patches.rockstargames.com
|
||||
@ -1,53 +0,0 @@
|
||||
# DNS Generation Scripts
|
||||
|
||||
## Introduction
|
||||
|
||||
The respective shell scripts contained within this directory can be utilised to generate application specific compliant
|
||||
configuration which can be utilised with:
|
||||
|
||||
* AdGuard Home
|
||||
* BIND9
|
||||
* Dnsmasq/Pi-hole
|
||||
* Squid
|
||||
* Unbound
|
||||
|
||||
## Usage
|
||||
|
||||
1. Copy `config.example.json` to `config.json`.
|
||||
2. Modify `config.json` to include your Cacheserver's IP(s) and the CDNs you plan to cache.
|
||||
|
||||
The following example assumes a single shared Cacheserver IP:
|
||||
```json
|
||||
{
|
||||
"combined_output": false,
|
||||
"ips": {
|
||||
"monolithic": ["10.10.10.200"]
|
||||
},
|
||||
"cache_domains": {
|
||||
"blizzard": "monolithic",
|
||||
"epicgames": "monolithic",
|
||||
"nintendo": "monolithic",
|
||||
"origin": "monolithic",
|
||||
"riot": "monolithic",
|
||||
"sony": "monolithic",
|
||||
"steam": "monolithic",
|
||||
"uplay": "monolithic",
|
||||
"wsus": "monolithic"
|
||||
}
|
||||
}
|
||||
```
|
||||
3. Run generation script relative to your DNS implementation: `bash create-dnsmasq.sh`.
|
||||
4. If `combined_output` is set to `true` this will result in a single output file: `lancache.conf` with all your enabled services (applies to Adguard Home, Dnsmasq or Unbound).
|
||||
5. Copy files from `output/{adguardhome,dnsmasq,rpz,squid,unbound}/*` to the respective locations for Dnsmasq/Unbound.
|
||||
6. Restart the appropriate service.
|
||||
|
||||
### Notes for Dnsmasq users
|
||||
|
||||
**This also applies to users utilising the script alongside Pi-hole.**
|
||||
|
||||
Multi-IP Lancache setups are only supported with Dnsmasq or Pi-hole versions >= 2.86 or 2021.09 respectively.
|
||||
|
||||
### Notes for AdGuard Home users
|
||||
|
||||
1. Utilising `"combined_output": true` is more convenient.
|
||||
2. Once you have run the script and uploaded the file to the appropriate location, you should navigate to Adguard Home -> Filters -> DNS blocklists -> Add blocklist -> Add a custom list.
|
||||
@ -1,80 +0,0 @@
|
||||
#!/bin/bash
|
||||
basedir=".."
|
||||
outputdir="output/adguardhome"
|
||||
path="${basedir}/cache_domains.json"
|
||||
|
||||
export IFS=' '
|
||||
|
||||
test=$(which jq);
|
||||
if [ $? -gt 0 ] ; then
|
||||
echo "This script requires jq to be installed."
|
||||
echo "Your package manager should be able to find it"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
cachenamedefault="disabled"
|
||||
combinedoutput=$(jq -r ".combined_output" config.json)
|
||||
|
||||
while read line; do
|
||||
ip=$(jq ".ips[\"${line}\"]" config.json)
|
||||
declare "cacheip${line}"="${ip}"
|
||||
done <<< $(jq -r '.ips | to_entries[] | .key' config.json)
|
||||
|
||||
while read line; do
|
||||
name=$(jq -r ".cache_domains[\"${line}\"]" config.json)
|
||||
declare "cachename$line"="$name"
|
||||
done <<< $(jq -r '.cache_domains | to_entries[] | .key' config.json)
|
||||
|
||||
rm -rf ${outputdir}
|
||||
mkdir -p ${outputdir}
|
||||
while read entry; do
|
||||
unset cacheip
|
||||
unset cachename
|
||||
key=$(jq -r ".cache_domains[$entry].name" ${path})
|
||||
cachename="cachename${key}"
|
||||
if [ -z "${!cachename}" ]; then
|
||||
cachename="cachenamedefault"
|
||||
fi
|
||||
if [[ ${!cachename} == "disabled" ]]; then
|
||||
continue;
|
||||
fi
|
||||
cacheipname="cacheip${!cachename}"
|
||||
cacheip=$(jq -r 'if type == "array" then .[] else . end' <<< ${!cacheipname} | xargs)
|
||||
while read fileid; do
|
||||
while read filename; do
|
||||
destfilename=$(echo ${filename} | sed -e 's/txt/conf/')
|
||||
outputfile=${outputdir}/${destfilename}
|
||||
touch ${outputfile}
|
||||
while read fileentry; do
|
||||
# Ignore comments and newlines
|
||||
if [[ ${fileentry} == \#* ]] || [[ -z ${fileentry} ]]; then
|
||||
continue
|
||||
fi
|
||||
domainprefix="|"
|
||||
if [[ $fileentry =~ ^\*\. ]]; then
|
||||
domainprefix="||"
|
||||
fi
|
||||
parsed=$(echo ${fileentry} | sed -e "s/^\*\.//")
|
||||
if grep -q "${domainprefix}${parsed}^\$dnsrewrite" ${outputfile}; then
|
||||
continue
|
||||
fi
|
||||
for i in ${cacheip}; do
|
||||
echo "${domainprefix}${parsed}^\$dnsrewrite=${i}" >> ${outputfile}
|
||||
done
|
||||
done <<< $(cat ${basedir}/$filename | sort);
|
||||
done <<< $(jq -r ".cache_domains[${entry}].domain_files[${fileid}]" ${path})
|
||||
done <<< $(jq -r ".cache_domains[${entry}].domain_files | to_entries[] | .key" ${path})
|
||||
done <<< $(jq -r '.cache_domains | to_entries[] | .key' ${path})
|
||||
|
||||
if [[ ${combinedoutput} == "true" ]]; then
|
||||
for file in ${outputdir}/*; do f=${file//${outputdir}\/} && f=${f//.conf} && echo "# ${f^}" >> ${outputdir}/lancache.conf && cat ${file} >> ${outputdir}/lancache.conf && rm ${file}; done
|
||||
fi
|
||||
|
||||
cat << EOF
|
||||
Configuration generation completed.
|
||||
|
||||
Please copy the following files:
|
||||
- ./${outputdir}/*.conf to /opt/adguardhome/work/userfilters/
|
||||
- Navigate to Adguard Home -> Filters -> DNS blocklists -> Add blocklist -> Add a custom list
|
||||
- Add list for each service or utilise the combined output for a single list
|
||||
EOF
|
||||
@ -1,76 +0,0 @@
|
||||
#!/bin/bash
|
||||
basedir=".."
|
||||
outputdir="output/dnsmasq"
|
||||
path="${basedir}/cache_domains.json"
|
||||
|
||||
export IFS=' '
|
||||
|
||||
test=$(which jq);
|
||||
if [ $? -gt 0 ] ; then
|
||||
echo "This script requires jq to be installed."
|
||||
echo "Your package manager should be able to find it"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
cachenamedefault="disabled"
|
||||
combinedoutput=$(jq -r ".combined_output" config.json)
|
||||
|
||||
while read -r line; do
|
||||
ip=$(jq ".ips[\"${line}\"]" config.json)
|
||||
declare "cacheip${line}"="${ip}"
|
||||
done <<< $(jq -r '.ips | to_entries[] | .key' config.json)
|
||||
|
||||
while read -r line; do
|
||||
name=$(jq -r ".cache_domains[\"${line}\"]" config.json)
|
||||
declare "cachename${line}"="${name}"
|
||||
done <<< $(jq -r '.cache_domains | to_entries[] | .key' config.json)
|
||||
|
||||
rm -rf ${outputdir}
|
||||
mkdir -p ${outputdir}
|
||||
while read -r entry; do
|
||||
unset cacheip
|
||||
unset cachename
|
||||
key=$(jq -r ".cache_domains[${entry}].name" ${path})
|
||||
cachename="cachename${key}"
|
||||
if [ -z "${!cachename}" ]; then
|
||||
cachename="cachenamedefault"
|
||||
fi
|
||||
if [[ ${!cachename} == "disabled" ]]; then
|
||||
continue;
|
||||
fi
|
||||
cacheipname="cacheip${!cachename}"
|
||||
cacheip=$(jq -r 'if type == "array" then .[] else . end' <<< ${!cacheipname} | xargs)
|
||||
while read -r fileid; do
|
||||
while read -r filename; do
|
||||
destfilename=$(echo ${filename} | sed -e 's/txt/conf/')
|
||||
outputfile=${outputdir}/${destfilename}
|
||||
touch ${outputfile}
|
||||
while read -r fileentry; do
|
||||
# Ignore comments, newlines and wildcards
|
||||
if [[ ${fileentry} == \#* ]] || [[ -z ${fileentry} ]]; then
|
||||
continue
|
||||
fi
|
||||
parsed=$(echo ${fileentry} | sed -e "s/^\*\.//")
|
||||
for i in ${cacheip}; do
|
||||
if ! grep -qx "address=/${parsed}/${i}" "${outputfile}"; then
|
||||
echo "address=/${parsed}/${i}" >> "${outputfile}"
|
||||
fi
|
||||
if ! grep -qx "local=/${parsed}/" "${outputfile}"; then
|
||||
echo "local=/${parsed}/" >> "${outputfile}"
|
||||
fi
|
||||
done
|
||||
done <<< $(cat ${basedir}/${filename} | sort);
|
||||
done <<< $(jq -r ".cache_domains[${entry}].domain_files[$fileid]" ${path})
|
||||
done <<< $(jq -r ".cache_domains[${entry}].domain_files | to_entries[] | .key" ${path})
|
||||
done <<< $(jq -r '.cache_domains | to_entries[] | .key' ${path})
|
||||
|
||||
if [[ ${combinedoutput} == "true" ]]; then
|
||||
for file in ${outputdir}/*; do f=${file//${outputdir}\/} && f=${f//.conf} && echo "# ${f^}" >> ${outputdir}/lancache.conf && cat ${file} >> ${outputdir}/lancache.conf && rm ${file}; done
|
||||
fi
|
||||
|
||||
cat << EOF
|
||||
Configuration generation completed.
|
||||
|
||||
Please copy the following files:
|
||||
- ./${outputdir}/*.conf to /etc/dnsmasq/dnsmasq.d/
|
||||
EOF
|
||||
@ -1,113 +0,0 @@
|
||||
#!/bin/bash
|
||||
basedir=".."
|
||||
outputdir="output/rpz"
|
||||
path="${basedir}/cache_domains.json"
|
||||
basedomain=${1:-lancache.net}
|
||||
|
||||
export IFS=' '
|
||||
|
||||
test=$(which jq);
|
||||
if [ $? -gt 0 ] ; then
|
||||
echo "This script requires jq to be installed."
|
||||
echo "Your package manager should be able to find it"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
cachenamedefault="disabled"
|
||||
|
||||
while read line; do
|
||||
ip=$(jq ".ips[\"${line}\"]" config.json)
|
||||
declare "cacheip${line}"="${ip}"
|
||||
done <<< $(jq -r '.ips | to_entries[] | .key' config.json)
|
||||
|
||||
while read line; do
|
||||
name=$(jq -r ".cache_domains[\"${line}\"]" config.json)
|
||||
declare "cachename${line}"="${name}"
|
||||
done <<< $(jq -r '.cache_domains | to_entries[] | .key' config.json)
|
||||
|
||||
rm -rf ${outputdir}
|
||||
mkdir -p ${outputdir}
|
||||
outputfile=${outputdir}/db.rpz.${basedomain}
|
||||
cat > ${outputfile} << EOF
|
||||
\$TTL 60 ; default TTL
|
||||
\$ORIGIN rpz.${basedomain}.
|
||||
@ SOA ns1.${basedomain}. admin.${basedomain}. (
|
||||
$(date +%Y%m%d01) ; serial
|
||||
604800 ; refresh (1 week)
|
||||
600 ; retry (10 mins)
|
||||
600 ; expire (10 mins)
|
||||
600 ; minimum (10 mins)
|
||||
)
|
||||
NS ns1.${basedomain}.
|
||||
NS ns2.${basedomain}.
|
||||
|
||||
EOF
|
||||
|
||||
while read entry; do
|
||||
unset cacheip
|
||||
unset cachename
|
||||
key=$(jq -r ".cache_domains[${entry}].name" ${path})
|
||||
cachename="cachename${key}"
|
||||
if [ -z "${!cachename}" ]; then
|
||||
cachename="cachenamedefault"
|
||||
fi
|
||||
if [[ ${!cachename} == "disabled" ]]; then
|
||||
continue;
|
||||
fi
|
||||
cacheipname="cacheip${!cachename}"
|
||||
cacheip=$(jq -r 'if type == "array" then .[] else . end' <<< ${!cacheipname} | xargs)
|
||||
while read fileid; do
|
||||
while read filename; do
|
||||
echo "" >> ${outputfile}
|
||||
echo "; $(echo ${filename} | sed -e 's/.txt$//')" >> ${outputfile}
|
||||
destfilename=$(echo ${filename} | sed -e 's/txt/conf/')
|
||||
while read fileentry; do
|
||||
# Ignore comments and newlines
|
||||
if [[ ${fileentry} == \#* ]] || [[ -z ${fileentry} ]]; then
|
||||
continue
|
||||
fi
|
||||
parsed=$(echo ${fileentry})
|
||||
if grep -qx "^\"${parsed}\". " ${outputfile}; then
|
||||
continue
|
||||
fi
|
||||
t=""
|
||||
for i in ${cacheip}; do
|
||||
# only one cname per domain is allowed
|
||||
if [[ ${t} = "CNAME" ]]; then
|
||||
continue
|
||||
fi
|
||||
# for cnames you must use a fqdn with trailing dot
|
||||
t="CNAME"
|
||||
if [[ ${i} =~ ^[0-9\.]+$ ]] ; then
|
||||
t="A"
|
||||
elif [[ ! ${i} =~ \.$ ]] ; then
|
||||
i="${i}."
|
||||
fi
|
||||
printf "%-50s IN %s %s\n" \
|
||||
"${parsed}" \
|
||||
"${t}" \
|
||||
"${i}" \
|
||||
>> ${outputfile}
|
||||
done
|
||||
done <<< $(cat ${basedir}/${filename} | sort);
|
||||
done <<< $(jq -r ".cache_domains[${entry}].domain_files[${fileid}]" ${path})
|
||||
done <<< $(jq -r ".cache_domains[${entry}].domain_files | to_entries[] | .key" ${path})
|
||||
done <<< $(jq -r '.cache_domains | to_entries[] | .key' ${path})
|
||||
|
||||
cat << EOF
|
||||
Configuration generation completed.
|
||||
|
||||
Please include the rpz zone in your bind configuration"
|
||||
- cp ${outputfile} /etc/bind
|
||||
- configure the zone and use it
|
||||
|
||||
options {
|
||||
[...]
|
||||
response-policy {zone "rpz.${basedomain}";};
|
||||
[...]
|
||||
}
|
||||
zone "rpz.$basedomain" {
|
||||
type master;
|
||||
file "/etc/bind/db.rpz.${basedomain}";
|
||||
};
|
||||
EOF
|
||||
@ -1,70 +0,0 @@
|
||||
#!/bin/bash
|
||||
basedir=".."
|
||||
outputdir="output/squid"
|
||||
path="${basedir}/cache_domains.json"
|
||||
REGEX="^\\*\\.(.*)$"
|
||||
|
||||
export IFS=' '
|
||||
|
||||
test=$(which jq);
|
||||
if [ $? -gt 0 ] ; then
|
||||
echo "This script requires jq to be installed."
|
||||
echo "Your package manager should be able to find it"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
cachenamedefault="disabled"
|
||||
|
||||
while read -r line; do
|
||||
name=$(jq -r ".cache_domains[\"${line}\"]" config.json)
|
||||
declare "cachename${line}"="${name}"
|
||||
done <<< $(jq -r '.cache_domains | to_entries[] | .key' config.json)
|
||||
|
||||
rm -rf ${outputdir}
|
||||
mkdir -p ${outputdir}
|
||||
while read -r entry; do
|
||||
unset cachename
|
||||
key=$(jq -r ".cache_domains[$entry].name" ${path})
|
||||
cachename="cachename${key}"
|
||||
if [ -z "${!cachename}" ]; then
|
||||
cachename="cachenamedefault"
|
||||
fi
|
||||
if [[ ${!cachename} == "disabled" ]]; then
|
||||
continue;
|
||||
fi
|
||||
while read -r fileid; do
|
||||
while read -r filename; do
|
||||
destfilename=$(echo ${!cachename}.txt)
|
||||
outputfile=${outputdir}/${destfilename}
|
||||
touch ${outputfile}
|
||||
while read -r fileentry; do
|
||||
# Ignore comments
|
||||
if [[ ${fileentry} == \#* ]] || [[ -z ${fileentry} ]]; then
|
||||
continue
|
||||
fi
|
||||
# Ha wildcards to squid wildcards
|
||||
parsed=$(echo ${fileentry} | sed -e "s/^\*\./\./")
|
||||
# If we have cdn.thing and *.cdn.thing in cache_domains
|
||||
# Squid requires ONLY cdn.thing
|
||||
#
|
||||
# If the fileentry starts with *.cdn.thing
|
||||
if [[ ${fileentry} =~ $REGEX ]]; then
|
||||
# Does the cache_domains file also contain cdn.thing
|
||||
grep "${BASH_REMATCH[1]}" ${basedir}/${filename} | grep -v "${fileentry}" > /dev/null
|
||||
if [[ $? -eq 0 ]]; then
|
||||
# Skip *.cdn.thing as cdn.thing will be collected earlier/later
|
||||
continue
|
||||
fi
|
||||
fi
|
||||
echo "${parsed}" >> "${outputfile}"
|
||||
done <<< $(cat ${basedir}/${filename} | sort);
|
||||
done <<< $(jq -r ".cache_domains[${entry}].domain_files[${fileid}]" ${path})
|
||||
done <<< $(jq -r ".cache_domains[${entry}].domain_files | to_entries[] | .key" ${path})
|
||||
done <<< $(jq -r '.cache_domains | to_entries[] | .key' ${path})
|
||||
|
||||
cat << EOF
|
||||
Configuration generation completed.
|
||||
|
||||
Please copy the following files:
|
||||
- ./${outputdir}/*.txt to /etc/squid/domains/
|
||||
EOF
|
||||
@ -1,78 +0,0 @@
|
||||
#!/bin/bash
|
||||
basedir=".."
|
||||
outputdir="output/unbound"
|
||||
path="${basedir}/cache_domains.json"
|
||||
|
||||
export IFS=' '
|
||||
|
||||
test=$(which jq);
|
||||
if [ $? -gt 0 ] ; then
|
||||
echo "This script requires jq to be installed."
|
||||
echo "Your package manager should be able to find it"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
cachenamedefault="disabled"
|
||||
combinedoutput=$(jq -r ".combined_output" config.json)
|
||||
|
||||
while read line; do
|
||||
ip=$(jq ".ips[\"${line}\"]" config.json)
|
||||
declare "cacheip${line}"="${ip}"
|
||||
done <<< $(jq -r '.ips | to_entries[] | .key' config.json)
|
||||
|
||||
while read line; do
|
||||
name=$(jq -r ".cache_domains[\"${line}\"]" config.json)
|
||||
declare "cachename${line}"="${name}"
|
||||
done <<< $(jq -r '.cache_domains | to_entries[] | .key' config.json)
|
||||
|
||||
rm -rf ${outputdir}
|
||||
mkdir -p ${outputdir}
|
||||
while read entry; do
|
||||
unset cacheip
|
||||
unset cachename
|
||||
key=$(jq -r ".cache_domains[${entry}].name" ${path})
|
||||
cachename="cachename${key}"
|
||||
if [ -z "${!cachename}" ]; then
|
||||
cachename="cachenamedefault"
|
||||
fi
|
||||
if [[ ${!cachename} == "disabled" ]]; then
|
||||
continue;
|
||||
fi
|
||||
cacheipname="cacheip${!cachename}"
|
||||
cacheip=$(jq -r 'if type == "array" then .[] else . end' <<< ${!cacheipname} | xargs)
|
||||
while read fileid; do
|
||||
while read filename; do
|
||||
destfilename=$(echo ${filename} | sed -e 's/txt/conf/')
|
||||
outputfile=${outputdir}/${destfilename}
|
||||
touch ${outputfile}
|
||||
while read fileentry; do
|
||||
# Ignore comments and newlines
|
||||
if [[ ${fileentry} == \#* ]] || [[ -z ${fileentry} ]]; then
|
||||
continue
|
||||
fi
|
||||
parsed=$(echo ${fileentry} | sed -e "s/^\*\.//")
|
||||
if grep -qx " local-zone: \"${parsed}\" redirect" ${outputfile}; then
|
||||
continue
|
||||
fi
|
||||
if [[ $(head -n 1 ${outputfile}) != "server:" ]]; then
|
||||
echo "server:" >> ${outputfile}
|
||||
fi
|
||||
echo " local-zone: \"${parsed}\" redirect" >> ${outputfile}
|
||||
for i in ${cacheip}; do
|
||||
echo " local-data: \"${parsed} 30 IN A ${i}\"" >> ${outputfile}
|
||||
done
|
||||
done <<< $(cat ${basedir}/${filename} | sort);
|
||||
done <<< $(jq -r ".cache_domains[${entry}].domain_files[${fileid}]" ${path})
|
||||
done <<< $(jq -r ".cache_domains[${entry}].domain_files | to_entries[] | .key" ${path})
|
||||
done <<< $(jq -r '.cache_domains | to_entries[] | .key' ${path})
|
||||
|
||||
if [[ ${combinedoutput} == "true" ]]; then
|
||||
for file in ${outputdir}/*; do f=${file//${outputdir}\/} && f=${f//.conf} && echo "# ${f^}" >> ${outputdir}/lancache.conf && cat ${file} >> ${outputdir}/lancache.conf && rm ${file}; done
|
||||
fi
|
||||
|
||||
cat << EOF
|
||||
Configuration generation completed.
|
||||
|
||||
Please copy the following files:
|
||||
- ./${outputdir}/*.conf to /etc/unbound/unbound.conf.d/
|
||||
EOF
|
||||
16
sony.txt
16
sony.txt
@ -1,16 +0,0 @@
|
||||
gs2.ww.prod.dl.playstation.net
|
||||
*.gs2.ww.prod.dl.playstation.net
|
||||
*.gs2.sonycoment.loris-e.llnwd.net
|
||||
gs2-ww-prod.psn.akadns.net
|
||||
*.gs2-ww-prod.psn.akadns.net
|
||||
gs2.ww.prod.dl.playstation.net.edgesuite.net
|
||||
*.gs2.ww.prod.dl.playstation.net.edgesuite.net
|
||||
playstation4.sony.akadns.net
|
||||
theia.dl.playstation.net
|
||||
tmdb.np.dl.playstation.net
|
||||
gs-sec.ww.np.dl.playstation.net
|
||||
uef.np.dl.playstation.net
|
||||
gst.prod.dl.playstation.net
|
||||
vulcan.dl.playstation.net
|
||||
sgst.prod.dl.playstation.net
|
||||
psnobj.prod.dl.playstation.net
|
||||
@ -1 +0,0 @@
|
||||
patch-dl.ffxiv.com
|
||||
@ -1 +0,0 @@
|
||||
content.warframe.com
|
||||
@ -1,16 +0,0 @@
|
||||
dl2.wargaming.net
|
||||
wg.gcdn.co
|
||||
wgus-wotasia.wargaming.net
|
||||
dl-wot-ak.wargaming.net
|
||||
dl-wot-gc.wargaming.net
|
||||
dl-wot-se.wargaming.net
|
||||
dl-wot-cdx.wargaming.net
|
||||
dl-wows-ak.wargaming.net
|
||||
dl-wows-gc.wargaming.net
|
||||
dl-wows-se.wargaming.net
|
||||
dl-wows-cdx.wargaming.net
|
||||
dl-wowp-ak.wargaming.net
|
||||
dl-wowp-gc.wargaming.net
|
||||
dl-wowp-se.wargaming.net
|
||||
dl-wowp-cdx.wargaming.net
|
||||
wgus-woteu.wargaming.net
|
||||
@ -1,11 +0,0 @@
|
||||
*.windowsupdate.com
|
||||
*.dl.delivery.mp.microsoft.com
|
||||
dl.delivery.mp.microsoft.com
|
||||
*.update.microsoft.com
|
||||
*.do.dsp.mp.microsoft.com
|
||||
*.microsoft.com.edgesuite.net
|
||||
amupdatedl.microsoft.com
|
||||
amupdatedl2.microsoft.com
|
||||
amupdatedl3.microsoft.com
|
||||
amupdatedl4.microsoft.com
|
||||
amupdatedl5.microsoft.com
|
||||
@ -1,7 +0,0 @@
|
||||
assets1.xboxlive.com
|
||||
assets2.xboxlive.com
|
||||
xbox-mbr.xboxlive.com
|
||||
assets1.xboxlive.com.nsatc.net
|
||||
xvcf1.xboxlive.com
|
||||
xvcf2.xboxlive.com
|
||||
d1.xboxlive.com
|
||||
Reference in New Issue
Block a user