Slow Loading

[Oreo]

Is anyone else having problems? E is taking several minutes for a page to load. Not so on other sites. I already cleaned my cookies.

[Nico]

I had this problem as well, but it seems to be good now.

[King Serperior]

Things are loading slowly on my end as well, but I've chalked it up as my having to use this old laptop while I wait for my new one. 

[Oreo]

Still slow here.

took three minutes to edit this.

[Rhedyn]

Yeah, it's loading really slow for me too. It took me two attempts to log in and just trying to reply to this topic was a bit of a mission.

Edit: Seems to be better for me now.

[Oreo]

Better for me now too.

[Cycle]

Nuts.  Ya'll must be hogging the internet tubes because now my posts are uberslowed.  :(

[Far eyes]

There was a bad gateway 502 showing up for a bit seems to be working now

[Vekseid]

There was a bad gateway 502 showing up for a bit seems to be working now

That was me trying to fix it.

I'm still at a bit of a loss, as the bug was apparently affecting the slave server too. Still not sure how that happened.

[AmberStarfire]

It was running slowly for me on and off all day today. It hasn't just been in the last hour or so.

It slowed right down again (and seemed it would time out) after the site came back up, but now it's running faster again.

[Vekseid]

It was running slowly for me on and off all day today. It hasn't just been in the last hour or so.

It slowed right down again (and seemed it would time out) after the site came back up, but now it's running faster again.

Yeah, am keeping an eye on things.

The replication server was exhausting a resource and it's non-trivial to tell what. We're fine memory, bandwidth, disk i/o, and cpu-wise. Might have been hitting SOMAXCONN, but that seems really odd to be happening with replication.

[AmberStarfire]

Is there anything else in an error log that could help narrow it down?

[Vekseid]

Whatever it was, it didn't even register as system load.

I mostly figured it out because it was slowing the slave to a crawl too. Patching MySQL seems to have worked for now, but I'm not sure why it was able to affect anything else.

[AmberStarfire]

If changing the SQL helped, that suggests it may be database related and not server related. Yet it's a resource issue.

Could it have to do with the number of connections to the database at the same time?

[Vekseid]

...

Code Select Expand


./tuning-primer.sh

        -- MYSQL PERFORMANCE TUNING PRIMER --
             - By: Matthew Montgomery -

MySQL Version 5.5.37-0+wheezy1-log x86_64

Uptime = 0 days 4 hrs 8 min 45 sec
Avg. qps = 694
Total Questions = 10363562
Threads Connected = 6

Warning: Server has not been running for at least 48hrs.
It may not be safe to use these recommendations

To find out more information on how each of these
runtime variables effects performance visit:
http://dev.mysql.com/doc/refman/5.5/en/server-system-variables.html
Visit http://www.mysql.com/products/enterprise/advisors.html
for info about MySQL's Enterprise Monitoring and Advisory Service

SLOW QUERIES
The slow query log is enabled.
Current long_query_time = 1.000000 sec.
You have 102 out of 10363639 that take longer than 1.000000 sec. to complete
Your long_query_time seems to be fine

BINARY UPDATE LOG
The binary update log is enabled

WORKER THREADS
Current thread_cache_size = 256
Current threads_cached = 14
Current threads_per_sec = 0
Historic threads_per_sec = 0
Your thread_cache_size is fine

MAX CONNECTIONS
Current max_connections = 256
Current threads_connected = 7
Historic max_used_connections = 18
The number of used connections is 7% of the configured maximum.
You are using less than 10% of your configured max_connections.
Lowering max_connections could help to avoid an over-allocation of memory
See "MEMORY USAGE" section to make sure you are not over-allocating

INNODB STATUS
Current InnoDB index space = 2.44 G
Current InnoDB data space = 21.37 G
Current InnoDB buffer pool free = 0 %
Current innodb_buffer_pool_size = 24.00 G
Depending on how much space your innodb indexes take up it may be safe
to increase this value to up to 2 / 3 of total system memory

MEMORY USAGE
Max Memory Ever Allocated : 24.20 G
Configured Max Per-thread Buffers : 832 M
Configured Max Global Buffers : 24.14 G
Configured Max Memory Limit : 24.95 G
Physical Memory : 31.46 G
Max memory limit seem to be within acceptable norms

KEY BUFFER
Current MyISAM index space = 22 M
Current key_buffer_size = 16 M
Key cache miss rate is 1 : 21
Key buffer free ratio = 31 %
Your key_buffer_size seems to be too high.
Perhaps you can use these resources elsewhere

QUERY CACHE
Query cache is enabled
Current query_cache_size = 128 M
Current query_cache_used = 27 M
Current query_cache_limit = 256 K
Current Query cache Memory fill ratio = 21.16 %
Current query_cache_min_res_unit = 4 K
Your query_cache_size seems to be too high.
Perhaps you can use these resources elsewhere
MySQL won't cache query results that are larger than query_cache_limit in size

SORT OPERATIONS
Current sort_buffer_size = 256 K
Current read_rnd_buffer_size = 2 M
Sort buffer seems to be fine

JOINS
Current join_buffer_size = 260.00 K
You have had 6845 queries where a join could not use an index properly
You should enable "log-queries-not-using-indexes"
Then look for non indexed joins in the slow query log.
If you are unable to optimize your queries you may want to increase your
join_buffer_size to accommodate larger joins in one pass.

Note! This script will still suggest raising the join_buffer_size when
ANY joins not using indexes are found.

OPEN FILES LIMIT
Current open_files_limit = 33034 files
The open_files_limit should typically be set to at least 2x-3x
that of table_cache if you have heavy MyISAM usage.
Your open_files_limit value seems to be fine

TABLE CACHE
Current table_open_cache = 16384 tables
Current table_definition_cache = 4096 tables
You have a total of 363 tables
You have 474 open tables.
The table_cache value seems to be fine

TEMP TABLES
Current max_heap_table_size = 32 M
Current tmp_table_size = 32 M
Of 23668 temp tables, 17% were created on disk
Created disk tmp tables ratio seems fine

TABLE SCANS
Current read_buffer_size = 256 K
Current table scan ratio = 3372 : 1
read_buffer_size seems to be fine

TABLE LOCKING
Current Lock Wait ratio = 1 : 1259179
Your table locking seems to be fine


As I mentioned, it's something to do with replication itself. sar doesn't report anything, MySQL doesn't report anything. No connection backlog, and the firewall isn't on eth1. Yet as you said, the site was still slow between the time I booted MySQL on the master here and the time I rebooted it on the slave.

Unfortunately, I don't have access to IPMI at the moment (host is putting them behind a VPN... finally), so I'm loathe to do anything too invasive like messing with the TCP stack to see if the cause was somewhere in there.

[AmberStarfire]

There's a lot that goes beyond my knowledge, at least without referring to some books/the web. I'm curious as to why you have MyISAM showing under key buffer when you have InnoDB as your storage engine, but perhaps it's meant to be that way. InnoDB became the default with MySQL 5.5 and up but there are sometimes reasons to use MyISAM.

I was looking at the release notes for the version of MySQL you have and there are some bugs that might be somehow linked:
http://dev.mysql.com/doc/relnotes/mysql/5.5/en/news-5-5-37.html

[Edit] Actually those seem to be bugs fixed from the previous version but the link for the next version might go into the bugs with 5.5.37.

Also, re: the part about joins:

"JOINS
Current join_buffer_size = 260.00 K
You have had 6845 queries where a join could not use an index properly
You should enable "log-queries-not-using-indexes"
Then look for non indexed joins in the slow query log.
If you are unable to optimize your queries you may want to increase your
join_buffer_size to accommodate larger joins in one pass."

That's a lot of joins that aren't using an index properly. MyISAM doesn't support foreign keys (if I'm not mistaken), so that could be interfering with joins if some tables are using MyISAM and others are InnoDB. However, it might not be an issue/problem/anything like that at all. It may be intended to be the way it is.

[AmberStarfire]

I just got that bad gateway error again, but when I hit back and tried to resend my post, I saw it had gone through the first time.