advertising contact : info@nujetz.com
Frontend server migration - E-Hentai Forums
 

Welcome Guest ( Log In | Register )

2 Pages V  1 2 >  
Closed TopicStart new topic
> Frontend server migration

 
post Nov 11 2020, 11:36
Post #1
Tenboro

Admin


清く正しく


We will be migrating various frontend services to a set of shiny new servers* over the next several days. This post will be updated as the various stuff is moved.

If you encounter any new issues after the moves, please post about it here.


2020-11-11

- Thumbnails and static content has been (mostly) migrated.

The primary thumbnail servers now have a lot more capacity and all storage is now local, which means they no longer have to fetch any content from backend servers. This simplifies things a lot, and also improves performance.

Full IPv6 support has also been added for this.


2020-11-12

- Redirected all the remaining thumbnail server dependencies.

- Restored failover functionality for the bits that haven't had it since last summer.


2020-11-16

- The RPC/API frontends and failovers have been swapped over. (These are used for AJAX/inline loading requests.)


2020-11-17

- The main EH frontends and failovers have been swapped over.

This completes the migration from the old frontends, but there are still a couple of other services (archiving and the tracker) that will be moved.


2020-11-19

- The new archive servers have gone live.

The new archivers have both HTTPS and IPv6 support, and also have an order of a magnitude more bandwidth available.

Note that previously generated archives will still be served from the old servers for some time.

(While the available bandwidth is much larger, it is also shared with the rest of the frontend services in the new setup, so we may add a restriction that free downloads are not available around the peak traffic hours (at least during weekends) - this is however not currently active.)


2020-11-23

- The new tracker servers are now live.

The new trackers also add IPv6 support both on the network as well as on the tracker software itself, and should (hopefully) comply with the spec in [bittorrent.org] BEP7.

(Update: There were some issues with stats gathering after the update due to some changes in default behavior in new versions of underlying software, this should be fixed now.)

This completes the deployment of the new frontends, at least for now.


2020-11-29

- The old archiver servers have now been fully phased out.


* For the technically minded, there are four new physical servers that run 12 VMs each, with the following specs:

Dual Xeon Gold 5118
192 GB RAM
6 TB SSD storage
36TB HDD storage
10 Gbps uplinks
User is offlineProfile CardPM
Go to the top of the page
+Quote Post

 
post Nov 11 2020, 15:21
Post #2
darkknightx2



Casual Poster
****
Group: Members
Posts: 262
Joined: 11-April 11
Level 64 (Master)


I feel this site like a home for me...

thankyou for everything you do (IMG:[invalid] style_emoticons/default/smile.gif)
User is offlineProfile CardPM
Go to the top of the page
+Quote Post

 
post Nov 11 2020, 15:29
Post #3
moonflow



Active Poster
*******
Group: Gold Star Club
Posts: 1,483
Joined: 17-September 06
Level 500 (Dovahkiin)


Wow, didn't expect that many changes in such a short time frame. Thanks, Tenboro!
User is offlineProfile CardPM
Go to the top of the page
+Quote Post

 
post Nov 11 2020, 16:27
Post #4
dnszkdl



Veteran Poster
********
Group: Catgirl Camarilla
Posts: 2,726
Joined: 24-April 15
Level 500 (Ponyslayer)


Nice work!
User is offlineProfile CardPM
Go to the top of the page
+Quote Post

 
post Nov 11 2020, 16:35
Post #5
Shana



Shana is the source of my life.
****
Group: Gold Star Club
Posts: 387
Joined: 6-November 09
Level 500 (Ponyslayer)


Great work!
User is offlineProfile CardPM
Go to the top of the page
+Quote Post

 
post Nov 12 2020, 01:02
Post #6
blue penguin



in umbra, igitur, pugnabimus
***********
Group: Gold Star Club
Posts: 10,043
Joined: 24-March 12
Level 500 (Godslayer)


Ooooh, it explains the long DNS waits I'm getting.

In english: expect the site to load a tad slower over the next couple of days as nonauthoritative proxies update themselves.

Great deal of work Tenb!
User is offlineProfile CardPM
Go to the top of the page
+Quote Post

 
post Nov 12 2020, 01:07
Post #7
Nezu



Rat
********
Group: Catgirl Camarilla
Posts: 3,835
Joined: 29-January 12
Level 500 (Ponyslayer)


QUOTE(Tenboro @ Nov 11 2020, 09:36) *

... For the technically minded, there are four new physical servers which will eventually run 12 VMs each...


Cool! Is that 4x12 VMs for thumbnails/static content alone, or will these servers be shouldering some of the load for other parts of the site too?

And, how much of a capacity increase is this expected to be?

This post has been edited by lestion: Nov 12 2020, 01:08
User is offlineProfile CardPM
Go to the top of the page
+Quote Post

 
post Nov 12 2020, 09:01
Post #8
Tenboro

Admin


清く正しく


QUOTE(lestion @ Nov 12 2020, 00:07) *

Cool! Is that 4x12 VMs for thumbnails/static content alone, or will these servers be shouldering some of the load for other parts of the site too?


Static content is just 1 VM per physical server, the rest is for other things.

QUOTE(lestion @ Nov 12 2020, 00:07) *
And, how much of a capacity increase is this expected to be?


Hard to say exactly, since there's different kinds of "capacity" like actual storage space, CPU and bandwidth, but at least twice for all of those. The most important thing is that they can store all the thumbnails locally. Previously, the "combine" thumbnails used for small thumbs were reverse proxied from some other servers.
User is offlineProfile CardPM
Go to the top of the page
+Quote Post

 
post Nov 12 2020, 15:26
Post #9
O_Sakana_Tengoku



Newcomer
**
Group: Gold Star Club
Posts: 63
Joined: 27-February 15
Level 417 (Godslayer)


Nice work!
User is offlineProfile CardPM
Go to the top of the page
+Quote Post

 
post Nov 12 2020, 22:56
Post #10
Forth_Lancer



Incognito Bounty Translator
************
Group: Gold Star Club
Posts: 40,588
Joined: 11-September 08
Level 500 (Dovahkiin)


Just wondering, where the money comes from... (IMG:[invalid] style_emoticons/default/biggrin.gif)
User is offlineProfile CardPM
Go to the top of the page
+Quote Post

 
post Nov 13 2020, 10:34
Post #11
Tenboro

Admin


清く正しく


QUOTE(Forth_Lancer @ Nov 12 2020, 21:56) *

Just wondering, where the money comes from... (IMG:[invalid] style_emoticons/default/biggrin.gif)


[v.redd.it] https://v.redd.it/2z4ol3wfbuy51

Seriously though, the new servers aren't more expensive than the old ones overall, seeing as there are four new ones that'll replace six old ones + offload some tasks from some other servers.
User is offlineProfile CardPM
Go to the top of the page
+Quote Post

 
post Nov 14 2020, 03:53
Post #12
ddvd



Lurker
Group: Gold Star Club
Posts: 8
Joined: 16-October 19
Level 221 (Lord)


QUOTE(Tenboro @ Nov 13 2020, 03:34) *

[v.redd.it] https://v.redd.it/2z4ol3wfbuy51

Seriously though, the new servers aren't more expensive than the old ones overall, seeing as there are four new ones that'll replace six old ones + offload some tasks from some other servers.


LMAO
User is offlineProfile CardPM
Go to the top of the page
+Quote Post

 
post Nov 14 2020, 14:38
Post #13
NerfThis



Active Poster
*******
Group: Catgirl Camarilla
Posts: 2,467
Joined: 3-February 14
Level 500 (Ponyslayer)


wow
User is offlineProfile CardPM
Go to the top of the page
+Quote Post

 
post Nov 17 2020, 17:16
Post #14
nasu



さき★すかん
********
Group: Gold Star Club
Posts: 2,507
Joined: 13-June 16
Level 410 (Godslayer)


QUOTE(Tenboro @ Nov 11 2020, 09:36) *

2020-11-17

- The main EH frontends and failovers have been swapped over.

This completes the migration from the old frontends, but there are still a couple of other services (archiving and the tracker) that will be moved.

I was saying this morning how certain pages seem to load faster but I don't even know if it's placebo anymore considering pages normally loaded pretty much instantly for me anyways.

Either way, thanks for the hard work (IMG:[invalid] style_emoticons/default/biggrin.gif)
User is online!Profile CardPM
Go to the top of the page
+Quote Post

 
post Nov 17 2020, 19:37
Post #15
Wayward_Vagabond



ii-Kagen ni Shiro.
*********
Group: Gold Star Club
Posts: 6,304
Joined: 22-March 09
Level 367 (Dovahkiin)


It's been said before, but thanks Tenboro, for your service to this place. Nice to hear stuff is still ticking along and growing.
User is offlineProfile CardPM
Go to the top of the page
+Quote Post

 
post Nov 18 2020, 13:22
Post #16
TkzcM



root@Tkzc:~# rm -rf /*
***
Group: Gold Star Club
Posts: 158
Joined: 21-January 15
Level 164 (Lord)


QUOTE(Tenboro @ Nov 13 2020, 16:34) *

[v.redd.it] https://v.redd.it/2z4ol3wfbuy51

Seriously though, the new servers aren't more expensive than the old ones overall, seeing as there are four new ones that'll replace six old ones + offload some tasks from some other servers.


https://www.youtube.com/watch?v=UG7zLhEWanc

(IMG:[invalid] style_emoticons/default/cool.gif)

Thanks for your great work.

This post has been edited by TkzcM: Nov 18 2020, 13:23
User is offlineProfile CardPM
Go to the top of the page
+Quote Post

 
post Nov 22 2020, 16:12
Post #17
derpderp2



Casual Poster
***
Group: Gold Star Club
Posts: 215
Joined: 30-August 12
Level 457 (Godslayer)


thanks for all the work tenboro (IMG:[invalid] style_emoticons/default/smile.gif)
User is offlineProfile CardPM
Go to the top of the page
+Quote Post

 
post Nov 23 2020, 14:15
Post #18
Tenboro

Admin


清く正しく


As mentioned in the updated first post, the archivers and trackers have now both been deployed to the new servers.

Seeing as they both run on new major versions of most everything and use a brand new database connector/layer to support doing so, there might still be some remaining subtle issues caused by that. Let me know if you spot any.

IPv6 support for the tracker also required code changes in the tracker itself to support this, so let me know if there are any issues with this as well.
User is offlineProfile CardPM
Go to the top of the page
+Quote Post

 
post Nov 28 2020, 00:13
Post #19
uareader



Critter
*********
Group: Catgirl Camarilla
Posts: 5,533
Joined: 1-September 14
Level 500 (Ponyslayer)


I don't know what happened, but something is wrong right now.
Wanted to load the news page to play a last random encounter today, but it just won't load.
I hope it's not something that went wrong from from this migration.

edit: yeah, it was over https, it was from a bookmark, so the url could not be wrong. It always timed out. It's back to normal this morning of course, but it was really serious back then (IMG:[invalid] style_emoticons/default/sad.gif)

This post has been edited by uareader: Nov 28 2020, 07:49
User is offlineProfile CardPM
Go to the top of the page
+Quote Post

 
post Nov 28 2020, 00:24
Post #20
nasu



さき★すかん
********
Group: Gold Star Club
Posts: 2,507
Joined: 13-June 16
Level 410 (Godslayer)


QUOTE(uareader @ Nov 27 2020, 22:13) *

I don't know what happened, but something is wrong right now.
Wanted to load the news page to play a last random encounter today, but it just won't load.
I hope it's not something that went wrong from from this migration.

Works fine for me, are you loading over https?
User is online!Profile CardPM
Go to the top of the page
+Quote Post


2 Pages V  1 2 >
Closed TopicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 


Lo-Fi Version Time is now: 29th March 2024 - 01:35