Compare commits
14 Commits
anoa/devic
...
erikj/cach
| Author | SHA1 | Date | |
|---|---|---|---|
|
b1e498d6e1 | ||
|
|
9cca5ee743 | ||
|
|
24cc31ee96 | ||
|
|
3916e1b97a | ||
|
|
9cc168e42e | ||
|
|
41e4566682 | ||
|
|
234f55f3c4 | ||
|
|
4f5ca455bf | ||
|
|
83446a18fb | ||
|
|
271c322d08 | ||
|
|
c7376cdfe3 | ||
|
|
4fc53bf1fb | ||
|
|
f697b4b4a2 | ||
|
|
541f1b92d9 |
74
CHANGES.md
74
CHANGES.md
@@ -1,3 +1,77 @@
|
||||
Synapse 1.6.0rc1 (2019-11-20)
|
||||
=============================
|
||||
|
||||
Features
|
||||
--------
|
||||
|
||||
- Add federation support for cross-signing. ([\#5727](https://github.com/matrix-org/synapse/issues/5727))
|
||||
- Increase default room version from 4 to 5, thereby enforcing server key validity period checks. ([\#6220](https://github.com/matrix-org/synapse/issues/6220))
|
||||
- Add support for outbound http proxying via http_proxy/HTTPS_PROXY env vars. ([\#6238](https://github.com/matrix-org/synapse/issues/6238))
|
||||
- Implement label-based filtering on `/sync` and `/messages` ([MSC2326](https://github.com/matrix-org/matrix-doc/pull/2326)). ([\#6301](https://github.com/matrix-org/synapse/issues/6301), [\#6310](https://github.com/matrix-org/synapse/issues/6310), [\#6340](https://github.com/matrix-org/synapse/issues/6340))
|
||||
|
||||
|
||||
Bugfixes
|
||||
--------
|
||||
|
||||
- Fix LruCache callback deduplication for Python 3.8. Contributed by @V02460. ([\#6213](https://github.com/matrix-org/synapse/issues/6213))
|
||||
- Remove a room from a server's public rooms list on room upgrade. ([\#6232](https://github.com/matrix-org/synapse/issues/6232), [\#6235](https://github.com/matrix-org/synapse/issues/6235))
|
||||
- Delete keys from key backup when deleting backup versions. ([\#6253](https://github.com/matrix-org/synapse/issues/6253))
|
||||
- Make notification of cross-signing signatures work with workers. ([\#6254](https://github.com/matrix-org/synapse/issues/6254))
|
||||
- Fix exception when remote servers attempt to join a room that they're not allowed to join. ([\#6278](https://github.com/matrix-org/synapse/issues/6278))
|
||||
- Prevent errors from appearing on Synapse startup if `git` is not installed. ([\#6284](https://github.com/matrix-org/synapse/issues/6284))
|
||||
- Appservice requests will no longer contain a double slash prefix when the appservice url provided ends in a slash. ([\#6306](https://github.com/matrix-org/synapse/issues/6306))
|
||||
- Fix `/purge_room` admin API. ([\#6307](https://github.com/matrix-org/synapse/issues/6307))
|
||||
- Fix the `hidden` field in the `devices` table for SQLite versions prior to 3.23.0. ([\#6313](https://github.com/matrix-org/synapse/issues/6313))
|
||||
- Fix bug which casued rejected events to be persisted with the wrong room state. ([\#6320](https://github.com/matrix-org/synapse/issues/6320))
|
||||
- Fix bug where `rc_login` ratelimiting would prematurely kick in. ([\#6335](https://github.com/matrix-org/synapse/issues/6335))
|
||||
- Prevent the server taking a long time to start up when guest registration is enabled. ([\#6338](https://github.com/matrix-org/synapse/issues/6338))
|
||||
- Fix bug where upgrading a guest account to a full user would fail when account validity is enabled. ([\#6359](https://github.com/matrix-org/synapse/issues/6359))
|
||||
- Fix `to_device` stream ID getting reset every time Synapse restarts, which had the potential to cause unable to decrypt errors. ([\#6363](https://github.com/matrix-org/synapse/issues/6363))
|
||||
- Fix permission denied error when trying to generate a config file with the docker image. ([\#6389](https://github.com/matrix-org/synapse/issues/6389))
|
||||
|
||||
|
||||
Improved Documentation
|
||||
----------------------
|
||||
|
||||
- Contributor documentation now mentions script to run linters. ([\#6164](https://github.com/matrix-org/synapse/issues/6164))
|
||||
- Modify CAPTCHA_SETUP.md to update the terms `private key` and `public key` to `secret key` and `site key` respectively. Contributed by Yash Jipkate. ([\#6257](https://github.com/matrix-org/synapse/issues/6257))
|
||||
- Update `INSTALL.md` Email section to talk about `account_threepid_delegates`. ([\#6272](https://github.com/matrix-org/synapse/issues/6272))
|
||||
- Fix a small typo in `account_threepid_delegates` configuration option. ([\#6273](https://github.com/matrix-org/synapse/issues/6273))
|
||||
|
||||
|
||||
Internal Changes
|
||||
----------------
|
||||
|
||||
- Add a CI job to test the `synapse_port_db` script. ([\#6140](https://github.com/matrix-org/synapse/issues/6140), [\#6276](https://github.com/matrix-org/synapse/issues/6276))
|
||||
- Convert EventContext to an attrs. ([\#6218](https://github.com/matrix-org/synapse/issues/6218))
|
||||
- Move `persist_events` out from main data store. ([\#6240](https://github.com/matrix-org/synapse/issues/6240), [\#6300](https://github.com/matrix-org/synapse/issues/6300))
|
||||
- Reduce verbosity of user/room stats. ([\#6250](https://github.com/matrix-org/synapse/issues/6250))
|
||||
- Reduce impact of debug logging. ([\#6251](https://github.com/matrix-org/synapse/issues/6251))
|
||||
- Expose some homeserver functionality to spam checkers. ([\#6259](https://github.com/matrix-org/synapse/issues/6259))
|
||||
- Change cache descriptors to always return deferreds. ([\#6263](https://github.com/matrix-org/synapse/issues/6263), [\#6291](https://github.com/matrix-org/synapse/issues/6291))
|
||||
- Fix incorrect comment regarding the functionality of an `if` statement. ([\#6269](https://github.com/matrix-org/synapse/issues/6269))
|
||||
- Update CI to run `isort` over the `scripts` and `scripts-dev` directories. ([\#6270](https://github.com/matrix-org/synapse/issues/6270))
|
||||
- Replace every instance of `logger.warn` method with `logger.warning` as the former is deprecated. ([\#6271](https://github.com/matrix-org/synapse/issues/6271), [\#6314](https://github.com/matrix-org/synapse/issues/6314))
|
||||
- Port replication http server endpoints to async/await. ([\#6274](https://github.com/matrix-org/synapse/issues/6274))
|
||||
- Port room rest handlers to async/await. ([\#6275](https://github.com/matrix-org/synapse/issues/6275))
|
||||
- Remove redundant CLI parameters on CI's `flake8` step. ([\#6277](https://github.com/matrix-org/synapse/issues/6277))
|
||||
- Port `federation_server.py` to async/await. ([\#6279](https://github.com/matrix-org/synapse/issues/6279))
|
||||
- Port receipt and read markers to async/wait. ([\#6280](https://github.com/matrix-org/synapse/issues/6280))
|
||||
- Split out state storage into separate data store. ([\#6294](https://github.com/matrix-org/synapse/issues/6294), [\#6295](https://github.com/matrix-org/synapse/issues/6295))
|
||||
- Refactor EventContext for clarity. ([\#6298](https://github.com/matrix-org/synapse/issues/6298))
|
||||
- Update the version of black used to 19.10b0. ([\#6304](https://github.com/matrix-org/synapse/issues/6304))
|
||||
- Add some documentation about worker replication. ([\#6305](https://github.com/matrix-org/synapse/issues/6305))
|
||||
- Move admin endpoints into separate files. Contributed by Awesome Technologies Innovationslabor GmbH. ([\#6308](https://github.com/matrix-org/synapse/issues/6308))
|
||||
- Document the use of `lint.sh` for code style enforcement & extend it to run on specified paths only. ([\#6312](https://github.com/matrix-org/synapse/issues/6312))
|
||||
- Add optional python dependencies and dependant binary libraries to snapcraft packaging. ([\#6317](https://github.com/matrix-org/synapse/issues/6317))
|
||||
- Remove the dependency on psutil and replace functionality with the stdlib `resource` module. ([\#6318](https://github.com/matrix-org/synapse/issues/6318), [\#6336](https://github.com/matrix-org/synapse/issues/6336))
|
||||
- Improve documentation for EventContext fields. ([\#6319](https://github.com/matrix-org/synapse/issues/6319))
|
||||
- Add some checks that we aren't using state from rejected events. ([\#6330](https://github.com/matrix-org/synapse/issues/6330))
|
||||
- Add continuous integration for python 3.8. ([\#6341](https://github.com/matrix-org/synapse/issues/6341))
|
||||
- Correct spacing/case of various instances of the word "homeserver". ([\#6357](https://github.com/matrix-org/synapse/issues/6357))
|
||||
- Temporarily blacklist the failing unit test PurgeRoomTestCase.test_purge_room. ([\#6361](https://github.com/matrix-org/synapse/issues/6361))
|
||||
|
||||
|
||||
Synapse 1.5.1 (2019-11-06)
|
||||
==========================
|
||||
|
||||
|
||||
12
INSTALL.md
12
INSTALL.md
@@ -133,9 +133,9 @@ sudo yum install libtiff-devel libjpeg-devel libzip-devel freetype-devel \
|
||||
sudo yum groupinstall "Development Tools"
|
||||
```
|
||||
|
||||
#### Mac OS X
|
||||
#### macOS
|
||||
|
||||
Installing prerequisites on Mac OS X:
|
||||
Installing prerequisites on macOS:
|
||||
|
||||
```
|
||||
xcode-select --install
|
||||
@@ -144,6 +144,14 @@ sudo pip install virtualenv
|
||||
brew install pkg-config libffi
|
||||
```
|
||||
|
||||
On macOS Catalina (10.15) you may need to explicitly install OpenSSL
|
||||
via brew and inform `pip` about it so that `psycopg2` builds:
|
||||
|
||||
```
|
||||
brew install openssl@1.1
|
||||
export LDFLAGS=-L/usr/local/Cellar/openssl\@1.1/1.1.1d/lib/
|
||||
```
|
||||
|
||||
#### OpenSUSE
|
||||
|
||||
Installing prerequisites on openSUSE:
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
Add federation support for cross-signing.
|
||||
@@ -1 +0,0 @@
|
||||
Add a CI job to test the `synapse_port_db` script.
|
||||
@@ -1 +0,0 @@
|
||||
Contributor documentation now mentions script to run linters.
|
||||
@@ -1 +0,0 @@
|
||||
Fix LruCache callback deduplication for Python 3.8. Contributed by @V02460.
|
||||
@@ -1 +0,0 @@
|
||||
Convert EventContext to an attrs.
|
||||
@@ -1 +0,0 @@
|
||||
Increase default room version from 4 to 5, thereby enforcing server key validity period checks.
|
||||
@@ -1 +0,0 @@
|
||||
Remove a room from a server's public rooms list on room upgrade.
|
||||
@@ -1 +0,0 @@
|
||||
Remove a room from a server's public rooms list on room upgrade.
|
||||
@@ -1 +0,0 @@
|
||||
Add support for outbound http proxying via http_proxy/HTTPS_PROXY env vars.
|
||||
@@ -1 +0,0 @@
|
||||
Move `persist_events` out from main data store.
|
||||
@@ -1 +0,0 @@
|
||||
Reduce verbosity of user/room stats.
|
||||
@@ -1 +0,0 @@
|
||||
Reduce impact of debug logging.
|
||||
@@ -1 +0,0 @@
|
||||
Delete keys from key backup when deleting backup versions.
|
||||
@@ -1 +0,0 @@
|
||||
Make notification of cross-signing signatures work with workers.
|
||||
@@ -1 +0,0 @@
|
||||
Modify CAPTCHA_SETUP.md to update the terms `private key` and `public key` to `secret key` and `site key` respectively. Contributed by Yash Jipkate.
|
||||
@@ -1 +0,0 @@
|
||||
Expose some homeserver functionality to spam checkers.
|
||||
@@ -1 +0,0 @@
|
||||
Change cache descriptors to always return deferreds.
|
||||
@@ -1 +0,0 @@
|
||||
Fix incorrect comment regarding the functionality of an `if` statement.
|
||||
@@ -1 +0,0 @@
|
||||
Update CI to run `isort` over the `scripts` and `scripts-dev` directories.
|
||||
@@ -1 +0,0 @@
|
||||
Replace every instance of `logger.warn` method with `logger.warning` as the former is deprecated.
|
||||
@@ -1 +0,0 @@
|
||||
Update `INSTALL.md` Email section to talk about `account_threepid_delegates`.
|
||||
@@ -1 +0,0 @@
|
||||
Fix a small typo in `account_threepid_delegates` configuration option.
|
||||
@@ -1 +0,0 @@
|
||||
Port replication http server endpoints to async/await.
|
||||
@@ -1 +0,0 @@
|
||||
Port room rest handlers to async/await.
|
||||
@@ -1 +0,0 @@
|
||||
Add a CI job to test the `synapse_port_db` script.
|
||||
@@ -1 +0,0 @@
|
||||
Remove redundant CLI parameters on CI's `flake8` step.
|
||||
@@ -1 +0,0 @@
|
||||
Fix exception when remote servers attempt to join a room that they're not allowed to join.
|
||||
@@ -1 +0,0 @@
|
||||
Port `federation_server.py` to async/await.
|
||||
@@ -1 +0,0 @@
|
||||
Port receipt and read markers to async/wait.
|
||||
@@ -1 +0,0 @@
|
||||
Prevent errors from appearing on Synapse startup if `git` is not installed.
|
||||
@@ -1 +0,0 @@
|
||||
Change cache descriptors to always return deferreds.
|
||||
@@ -1 +0,0 @@
|
||||
Split out state storage into separate data store.
|
||||
@@ -1 +0,0 @@
|
||||
Split out state storage into separate data store.
|
||||
@@ -1 +0,0 @@
|
||||
Refactor EventContext for clarity.
|
||||
@@ -1 +0,0 @@
|
||||
Move `persist_events` out from main data store.
|
||||
@@ -1 +0,0 @@
|
||||
Implement label-based filtering on `/sync` and `/messages` ([MSC2326](https://github.com/matrix-org/matrix-doc/pull/2326)).
|
||||
@@ -1 +0,0 @@
|
||||
Update the version of black used to 19.10b0.
|
||||
@@ -1 +0,0 @@
|
||||
Add some documentation about worker replication.
|
||||
@@ -1 +0,0 @@
|
||||
Appservice requests will no longer contain a double slash prefix when the appservice url provided ends in a slash.
|
||||
@@ -1 +0,0 @@
|
||||
Fix `/purge_room` admin API.
|
||||
@@ -1 +0,0 @@
|
||||
Implement label-based filtering on `/sync` and `/messages` ([MSC2326](https://github.com/matrix-org/matrix-doc/pull/2326)).
|
||||
@@ -1 +0,0 @@
|
||||
Document the use of `lint.sh` for code style enforcement & extend it to run on specified paths only.
|
||||
@@ -1 +0,0 @@
|
||||
Fix the `hidden` field in the `devices` table for SQLite versions prior to 3.23.0.
|
||||
@@ -1 +0,0 @@
|
||||
Replace every instance of `logger.warn` method with `logger.warning` as the former is deprecated.
|
||||
@@ -1 +0,0 @@
|
||||
Add optional python dependencies and dependant binary libraries to snapcraft packaging.
|
||||
@@ -1 +0,0 @@
|
||||
Remove the dependency on psutil and replace functionality with the stdlib `resource` module.
|
||||
@@ -1 +0,0 @@
|
||||
Improve documentation for EventContext fields.
|
||||
@@ -1 +0,0 @@
|
||||
Fix bug which casued rejected events to be persisted with the wrong room state.
|
||||
@@ -1 +0,0 @@
|
||||
Add some checks that we aren't using state from rejected events.
|
||||
@@ -1 +0,0 @@
|
||||
Remove the dependency on psutil and replace functionality with the stdlib `resource` module.
|
||||
@@ -1 +0,0 @@
|
||||
Prevent the server taking a long time to start up when guest registration is enabled.
|
||||
@@ -1 +0,0 @@
|
||||
Implement label-based filtering on `/sync` and `/messages` ([MSC2326](https://github.com/matrix-org/matrix-doc/pull/2326)).
|
||||
@@ -1 +0,0 @@
|
||||
Add continuous integration for python 3.8.
|
||||
@@ -1 +0,0 @@
|
||||
Correct spacing/case of various instances of the word "homeserver".
|
||||
@@ -1 +0,0 @@
|
||||
Fix bug where upgrading a guest account to a full user would fail when account validity is enabled.
|
||||
@@ -1 +0,0 @@
|
||||
Temporarily blacklist the failing unit test PurgeRoomTestCase.test_purge_room.
|
||||
1
changelog.d/6362.misc
Normal file
1
changelog.d/6362.misc
Normal file
@@ -0,0 +1 @@
|
||||
Clean up some unnecessary quotation marks around the codebase.
|
||||
@@ -1 +0,0 @@
|
||||
Fix `to_device` stream ID getting reset every time Synapse restarts, which had the potential to cause unable to decrypt errors.
|
||||
1
changelog.d/6388.doc
Normal file
1
changelog.d/6388.doc
Normal file
@@ -0,0 +1 @@
|
||||
Fix link in the user directory documentation.
|
||||
@@ -169,11 +169,11 @@ def run_generate_config(environ, ownership):
|
||||
# log("running %s" % (args, ))
|
||||
|
||||
if ownership is not None:
|
||||
args = ["su-exec", ownership] + args
|
||||
os.execv("/sbin/su-exec", args)
|
||||
|
||||
# make sure that synapse has perms to write to the data dir.
|
||||
subprocess.check_output(["chown", ownership, data_dir])
|
||||
|
||||
args = ["su-exec", ownership] + args
|
||||
os.execv("/sbin/su-exec", args)
|
||||
else:
|
||||
os.execv("/usr/local/bin/python", args)
|
||||
|
||||
|
||||
@@ -7,7 +7,6 @@ who are present in a publicly viewable room present on the server.
|
||||
|
||||
The directory info is stored in various tables, which can (typically after
|
||||
DB corruption) get stale or out of sync. If this happens, for now the
|
||||
solution to fix it is to execute the SQL here
|
||||
https://github.com/matrix-org/synapse/blob/master/synapse/storage/schema/delta/53/user_dir_populate.sql
|
||||
solution to fix it is to execute the SQL [here](../synapse/storage/data_stores/main/schema/delta/53/user_dir_populate.sql)
|
||||
and then restart synapse. This should then start a background task to
|
||||
flush the current tables and regenerate the directory.
|
||||
|
||||
@@ -36,7 +36,7 @@ try:
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
__version__ = "1.5.1"
|
||||
__version__ = "1.6.0rc1"
|
||||
|
||||
if bool(os.environ.get("SYNAPSE_TEST_PATCH_LOG_CONTEXTS", False)):
|
||||
# We import here so that we don't have to install a bunch of deps when
|
||||
|
||||
@@ -69,7 +69,7 @@ class FederationSenderSlaveStore(
|
||||
self.federation_out_pos_startup = self._get_federation_out_pos(db_conn)
|
||||
|
||||
def _get_federation_out_pos(self, db_conn):
|
||||
sql = "SELECT stream_id FROM federation_stream_position" " WHERE type = ?"
|
||||
sql = "SELECT stream_id FROM federation_stream_position WHERE type = ?"
|
||||
sql = self.database_engine.convert_param_style(sql)
|
||||
|
||||
txn = db_conn.cursor()
|
||||
|
||||
@@ -185,7 +185,7 @@ class ApplicationServiceApi(SimpleHttpClient):
|
||||
|
||||
if not _is_valid_3pe_metadata(info):
|
||||
logger.warning(
|
||||
"query_3pe_protocol to %s did not return a" " valid result", uri
|
||||
"query_3pe_protocol to %s did not return a valid result", uri
|
||||
)
|
||||
return None
|
||||
|
||||
|
||||
@@ -134,7 +134,7 @@ def _load_appservice(hostname, as_info, config_filename):
|
||||
for regex_obj in as_info["namespaces"][ns]:
|
||||
if not isinstance(regex_obj, dict):
|
||||
raise ValueError(
|
||||
"Expected namespace entry in %s to be an object," " but got %s",
|
||||
"Expected namespace entry in %s to be an object, but got %s",
|
||||
ns,
|
||||
regex_obj,
|
||||
)
|
||||
|
||||
@@ -170,7 +170,7 @@ class _RoomDirectoryRule(object):
|
||||
self.action = action
|
||||
else:
|
||||
raise ConfigError(
|
||||
"%s rules can only have action of 'allow'" " or 'deny'" % (option_name,)
|
||||
"%s rules can only have action of 'allow' or 'deny'" % (option_name,)
|
||||
)
|
||||
|
||||
self._alias_matches_all = alias == "*"
|
||||
|
||||
@@ -223,7 +223,7 @@ class ServerConfig(Config):
|
||||
self.federation_ip_range_blacklist.update(["0.0.0.0", "::"])
|
||||
except Exception as e:
|
||||
raise ConfigError(
|
||||
"Invalid range(s) provided in " "federation_ip_range_blacklist: %s" % e
|
||||
"Invalid range(s) provided in federation_ip_range_blacklist: %s" % e
|
||||
)
|
||||
|
||||
if self.public_baseurl is not None:
|
||||
@@ -787,14 +787,14 @@ class ServerConfig(Config):
|
||||
"--print-pidfile",
|
||||
action="store_true",
|
||||
default=None,
|
||||
help="Print the path to the pidfile just" " before daemonizing",
|
||||
help="Print the path to the pidfile just before daemonizing",
|
||||
)
|
||||
server_group.add_argument(
|
||||
"--manhole",
|
||||
metavar="PORT",
|
||||
dest="manhole",
|
||||
type=int,
|
||||
help="Turn on the twisted telnet manhole" " service on the given port.",
|
||||
help="Turn on the twisted telnet manhole service on the given port.",
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -44,7 +44,7 @@ class TransactionActions(object):
|
||||
response code and response body.
|
||||
"""
|
||||
if not transaction.transaction_id:
|
||||
raise RuntimeError("Cannot persist a transaction with no " "transaction_id")
|
||||
raise RuntimeError("Cannot persist a transaction with no transaction_id")
|
||||
|
||||
return self.store.get_received_txn_response(transaction.transaction_id, origin)
|
||||
|
||||
@@ -56,7 +56,7 @@ class TransactionActions(object):
|
||||
Deferred
|
||||
"""
|
||||
if not transaction.transaction_id:
|
||||
raise RuntimeError("Cannot persist a transaction with no " "transaction_id")
|
||||
raise RuntimeError("Cannot persist a transaction with no transaction_id")
|
||||
|
||||
return self.store.set_received_txn_response(
|
||||
transaction.transaction_id, origin, code, response
|
||||
|
||||
@@ -49,7 +49,7 @@ sent_pdus_destination_dist_count = Counter(
|
||||
|
||||
sent_pdus_destination_dist_total = Counter(
|
||||
"synapse_federation_client_sent_pdu_destinations:total",
|
||||
"" "Total number of PDUs queued for sending across all destinations",
|
||||
"Total number of PDUs queued for sending across all destinations",
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -84,7 +84,7 @@ class TransactionManager(object):
|
||||
txn_id = str(self._next_txn_id)
|
||||
|
||||
logger.debug(
|
||||
"TX [%s] {%s} Attempting new transaction" " (pdus: %d, edus: %d)",
|
||||
"TX [%s] {%s} Attempting new transaction (pdus: %d, edus: %d)",
|
||||
destination,
|
||||
txn_id,
|
||||
len(pdus),
|
||||
@@ -103,7 +103,7 @@ class TransactionManager(object):
|
||||
self._next_txn_id += 1
|
||||
|
||||
logger.info(
|
||||
"TX [%s] {%s} Sending transaction [%s]," " (PDUs: %d, EDUs: %d)",
|
||||
"TX [%s] {%s} Sending transaction [%s], (PDUs: %d, EDUs: %d)",
|
||||
destination,
|
||||
txn_id,
|
||||
transaction.transaction_id,
|
||||
|
||||
@@ -102,8 +102,9 @@ class AuthHandler(BaseHandler):
|
||||
login_types.append(t)
|
||||
self._supported_login_types = login_types
|
||||
|
||||
self._account_ratelimiter = Ratelimiter()
|
||||
self._failed_attempts_ratelimiter = Ratelimiter()
|
||||
# Ratelimiter for failed auth during UIA. Uses same ratelimit config
|
||||
# as per `rc_login.failed_attempts`.
|
||||
self._failed_uia_attempts_ratelimiter = Ratelimiter()
|
||||
|
||||
self._clock = self.hs.get_clock()
|
||||
|
||||
@@ -133,12 +134,38 @@ class AuthHandler(BaseHandler):
|
||||
|
||||
AuthError if the client has completed a login flow, and it gives
|
||||
a different user to `requester`
|
||||
|
||||
LimitExceededError if the ratelimiter's failed request count for this
|
||||
user is too high to proceed
|
||||
|
||||
"""
|
||||
|
||||
user_id = requester.user.to_string()
|
||||
|
||||
# Check if we should be ratelimited due to too many previous failed attempts
|
||||
self._failed_uia_attempts_ratelimiter.ratelimit(
|
||||
user_id,
|
||||
time_now_s=self._clock.time(),
|
||||
rate_hz=self.hs.config.rc_login_failed_attempts.per_second,
|
||||
burst_count=self.hs.config.rc_login_failed_attempts.burst_count,
|
||||
update=False,
|
||||
)
|
||||
|
||||
# build a list of supported flows
|
||||
flows = [[login_type] for login_type in self._supported_login_types]
|
||||
|
||||
result, params, _ = yield self.check_auth(flows, request_body, clientip)
|
||||
try:
|
||||
result, params, _ = yield self.check_auth(flows, request_body, clientip)
|
||||
except LoginError:
|
||||
# Update the ratelimite to say we failed (`can_do_action` doesn't raise).
|
||||
self._failed_uia_attempts_ratelimiter.can_do_action(
|
||||
user_id,
|
||||
time_now_s=self._clock.time(),
|
||||
rate_hz=self.hs.config.rc_login_failed_attempts.per_second,
|
||||
burst_count=self.hs.config.rc_login_failed_attempts.burst_count,
|
||||
update=True,
|
||||
)
|
||||
raise
|
||||
|
||||
# find the completed login type
|
||||
for login_type in self._supported_login_types:
|
||||
@@ -501,11 +528,8 @@ class AuthHandler(BaseHandler):
|
||||
multiple matches
|
||||
|
||||
Raises:
|
||||
LimitExceededError if the ratelimiter's login requests count for this
|
||||
user is too high too proceed.
|
||||
UserDeactivatedError if a user is found but is deactivated.
|
||||
"""
|
||||
self.ratelimit_login_per_account(user_id)
|
||||
res = yield self._find_user_id_and_pwd_hash(user_id)
|
||||
if res is not None:
|
||||
return res[0]
|
||||
@@ -572,8 +596,6 @@ class AuthHandler(BaseHandler):
|
||||
StoreError if there was a problem accessing the database
|
||||
SynapseError if there was a problem with the request
|
||||
LoginError if there was an authentication problem.
|
||||
LimitExceededError if the ratelimiter's login requests count for this
|
||||
user is too high too proceed.
|
||||
"""
|
||||
|
||||
if username.startswith("@"):
|
||||
@@ -581,8 +603,6 @@ class AuthHandler(BaseHandler):
|
||||
else:
|
||||
qualified_user_id = UserID(username, self.hs.hostname).to_string()
|
||||
|
||||
self.ratelimit_login_per_account(qualified_user_id)
|
||||
|
||||
login_type = login_submission.get("type")
|
||||
known_login_type = False
|
||||
|
||||
@@ -650,15 +670,6 @@ class AuthHandler(BaseHandler):
|
||||
if not known_login_type:
|
||||
raise SynapseError(400, "Unknown login type %s" % login_type)
|
||||
|
||||
# unknown username or invalid password.
|
||||
self._failed_attempts_ratelimiter.ratelimit(
|
||||
qualified_user_id.lower(),
|
||||
time_now_s=self._clock.time(),
|
||||
rate_hz=self.hs.config.rc_login_failed_attempts.per_second,
|
||||
burst_count=self.hs.config.rc_login_failed_attempts.burst_count,
|
||||
update=True,
|
||||
)
|
||||
|
||||
# We raise a 403 here, but note that if we're doing user-interactive
|
||||
# login, it turns all LoginErrors into a 401 anyway.
|
||||
raise LoginError(403, "Invalid password", errcode=Codes.FORBIDDEN)
|
||||
@@ -710,10 +721,6 @@ class AuthHandler(BaseHandler):
|
||||
Returns:
|
||||
Deferred[unicode] the canonical_user_id, or Deferred[None] if
|
||||
unknown user/bad password
|
||||
|
||||
Raises:
|
||||
LimitExceededError if the ratelimiter's login requests count for this
|
||||
user is too high too proceed.
|
||||
"""
|
||||
lookupres = yield self._find_user_id_and_pwd_hash(user_id)
|
||||
if not lookupres:
|
||||
@@ -742,7 +749,7 @@ class AuthHandler(BaseHandler):
|
||||
auth_api.validate_macaroon(macaroon, "login", user_id)
|
||||
except Exception:
|
||||
raise AuthError(403, "Invalid token", errcode=Codes.FORBIDDEN)
|
||||
self.ratelimit_login_per_account(user_id)
|
||||
|
||||
yield self.auth.check_auth_blocking(user_id)
|
||||
return user_id
|
||||
|
||||
@@ -912,35 +919,6 @@ class AuthHandler(BaseHandler):
|
||||
else:
|
||||
return defer.succeed(False)
|
||||
|
||||
def ratelimit_login_per_account(self, user_id):
|
||||
"""Checks whether the process must be stopped because of ratelimiting.
|
||||
|
||||
Checks against two ratelimiters: the generic one for login attempts per
|
||||
account and the one specific to failed attempts.
|
||||
|
||||
Args:
|
||||
user_id (unicode): complete @user:id
|
||||
|
||||
Raises:
|
||||
LimitExceededError if one of the ratelimiters' login requests count
|
||||
for this user is too high too proceed.
|
||||
"""
|
||||
self._failed_attempts_ratelimiter.ratelimit(
|
||||
user_id.lower(),
|
||||
time_now_s=self._clock.time(),
|
||||
rate_hz=self.hs.config.rc_login_failed_attempts.per_second,
|
||||
burst_count=self.hs.config.rc_login_failed_attempts.burst_count,
|
||||
update=False,
|
||||
)
|
||||
|
||||
self._account_ratelimiter.ratelimit(
|
||||
user_id.lower(),
|
||||
time_now_s=self._clock.time(),
|
||||
rate_hz=self.hs.config.rc_login_account.per_second,
|
||||
burst_count=self.hs.config.rc_login_account.burst_count,
|
||||
update=True,
|
||||
)
|
||||
|
||||
|
||||
@attr.s
|
||||
class MacaroonGenerator(object):
|
||||
|
||||
@@ -119,7 +119,7 @@ class DirectoryHandler(BaseHandler):
|
||||
if not service.is_interested_in_alias(room_alias.to_string()):
|
||||
raise SynapseError(
|
||||
400,
|
||||
"This application service has not reserved" " this kind of alias.",
|
||||
"This application service has not reserved this kind of alias.",
|
||||
errcode=Codes.EXCLUSIVE,
|
||||
)
|
||||
else:
|
||||
|
||||
@@ -96,7 +96,7 @@ def parse_boolean_from_args(args, name, default=None, required=False):
|
||||
return {b"true": True, b"false": False}[args[name][0]]
|
||||
except Exception:
|
||||
message = (
|
||||
"Boolean query parameter %r must be one of" " ['true', 'false']"
|
||||
"Boolean query parameter %r must be one of ['true', 'false']"
|
||||
) % (name,)
|
||||
raise SynapseError(400, message)
|
||||
else:
|
||||
|
||||
@@ -246,7 +246,7 @@ class HttpPusher(object):
|
||||
# fixed, we don't suddenly deliver a load
|
||||
# of old notifications.
|
||||
logger.warning(
|
||||
"Giving up on a notification to user %s, " "pushkey %s",
|
||||
"Giving up on a notification to user %s, pushkey %s",
|
||||
self.user_id,
|
||||
self.pushkey,
|
||||
)
|
||||
@@ -299,8 +299,7 @@ class HttpPusher(object):
|
||||
# for sanity, we only remove the pushkey if it
|
||||
# was the one we actually sent...
|
||||
logger.warning(
|
||||
("Ignoring rejected pushkey %s because we" " didn't send it"),
|
||||
pk,
|
||||
("Ignoring rejected pushkey %s because we didn't send it"), pk,
|
||||
)
|
||||
else:
|
||||
logger.info("Pushkey %s was rejected: removing", pk)
|
||||
|
||||
@@ -43,7 +43,7 @@ logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
MESSAGE_FROM_PERSON_IN_ROOM = (
|
||||
"You have a message on %(app)s from %(person)s " "in the %(room)s room..."
|
||||
"You have a message on %(app)s from %(person)s in the %(room)s room..."
|
||||
)
|
||||
MESSAGE_FROM_PERSON = "You have a message on %(app)s from %(person)s..."
|
||||
MESSAGES_FROM_PERSON = "You have messages on %(app)s from %(person)s..."
|
||||
@@ -55,7 +55,7 @@ MESSAGES_FROM_PERSON_AND_OTHERS = (
|
||||
"You have messages on %(app)s from %(person)s and others..."
|
||||
)
|
||||
INVITE_FROM_PERSON_TO_ROOM = (
|
||||
"%(person)s has invited you to join the " "%(room)s room on %(app)s..."
|
||||
"%(person)s has invited you to join the %(room)s room on %(app)s..."
|
||||
)
|
||||
INVITE_FROM_PERSON = "%(person)s has invited you to chat on %(app)s..."
|
||||
|
||||
|
||||
@@ -14,62 +14,39 @@
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
import hashlib
|
||||
import hmac
|
||||
import logging
|
||||
import platform
|
||||
import re
|
||||
|
||||
from six import text_type
|
||||
from six.moves import http_client
|
||||
|
||||
import synapse
|
||||
from synapse.api.constants import Membership, UserTypes
|
||||
from synapse.api.errors import Codes, NotFoundError, SynapseError
|
||||
from synapse.http.server import JsonResource
|
||||
from synapse.http.servlet import (
|
||||
RestServlet,
|
||||
assert_params_in_dict,
|
||||
parse_integer,
|
||||
parse_json_object_from_request,
|
||||
parse_string,
|
||||
)
|
||||
from synapse.http.servlet import RestServlet, parse_json_object_from_request
|
||||
from synapse.rest.admin._base import (
|
||||
assert_requester_is_admin,
|
||||
assert_user_is_admin,
|
||||
historical_admin_path_patterns,
|
||||
)
|
||||
from synapse.rest.admin.groups import DeleteGroupAdminRestServlet
|
||||
from synapse.rest.admin.media import ListMediaInRoom, register_servlets_for_media_repo
|
||||
from synapse.rest.admin.purge_room_servlet import PurgeRoomServlet
|
||||
from synapse.rest.admin.rooms import ShutdownRoomRestServlet
|
||||
from synapse.rest.admin.server_notice_servlet import SendServerNoticeServlet
|
||||
from synapse.rest.admin.users import UserAdminServlet
|
||||
from synapse.types import UserID, create_requester
|
||||
from synapse.util.async_helpers import maybe_awaitable
|
||||
from synapse.rest.admin.users import (
|
||||
AccountValidityRenewServlet,
|
||||
DeactivateAccountRestServlet,
|
||||
GetUsersPaginatedRestServlet,
|
||||
ResetPasswordRestServlet,
|
||||
SearchUsersRestServlet,
|
||||
UserAdminServlet,
|
||||
UserRegisterServlet,
|
||||
UsersRestServlet,
|
||||
WhoisRestServlet,
|
||||
)
|
||||
from synapse.util.versionstring import get_version_string
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class UsersRestServlet(RestServlet):
|
||||
PATTERNS = historical_admin_path_patterns("/users/(?P<user_id>[^/]*)$")
|
||||
|
||||
def __init__(self, hs):
|
||||
self.hs = hs
|
||||
self.auth = hs.get_auth()
|
||||
self.handlers = hs.get_handlers()
|
||||
|
||||
async def on_GET(self, request, user_id):
|
||||
target_user = UserID.from_string(user_id)
|
||||
await assert_requester_is_admin(self.auth, request)
|
||||
|
||||
if not self.hs.is_mine(target_user):
|
||||
raise SynapseError(400, "Can only users a local user")
|
||||
|
||||
ret = await self.handlers.admin_handler.get_users()
|
||||
|
||||
return 200, ret
|
||||
|
||||
|
||||
class VersionServlet(RestServlet):
|
||||
PATTERNS = (re.compile("^/_synapse/admin/v1/server_version$"),)
|
||||
|
||||
@@ -83,159 +60,6 @@ class VersionServlet(RestServlet):
|
||||
return 200, self.res
|
||||
|
||||
|
||||
class UserRegisterServlet(RestServlet):
|
||||
"""
|
||||
Attributes:
|
||||
NONCE_TIMEOUT (int): Seconds until a generated nonce won't be accepted
|
||||
nonces (dict[str, int]): The nonces that we will accept. A dict of
|
||||
nonce to the time it was generated, in int seconds.
|
||||
"""
|
||||
|
||||
PATTERNS = historical_admin_path_patterns("/register")
|
||||
NONCE_TIMEOUT = 60
|
||||
|
||||
def __init__(self, hs):
|
||||
self.handlers = hs.get_handlers()
|
||||
self.reactor = hs.get_reactor()
|
||||
self.nonces = {}
|
||||
self.hs = hs
|
||||
|
||||
def _clear_old_nonces(self):
|
||||
"""
|
||||
Clear out old nonces that are older than NONCE_TIMEOUT.
|
||||
"""
|
||||
now = int(self.reactor.seconds())
|
||||
|
||||
for k, v in list(self.nonces.items()):
|
||||
if now - v > self.NONCE_TIMEOUT:
|
||||
del self.nonces[k]
|
||||
|
||||
def on_GET(self, request):
|
||||
"""
|
||||
Generate a new nonce.
|
||||
"""
|
||||
self._clear_old_nonces()
|
||||
|
||||
nonce = self.hs.get_secrets().token_hex(64)
|
||||
self.nonces[nonce] = int(self.reactor.seconds())
|
||||
return 200, {"nonce": nonce}
|
||||
|
||||
async def on_POST(self, request):
|
||||
self._clear_old_nonces()
|
||||
|
||||
if not self.hs.config.registration_shared_secret:
|
||||
raise SynapseError(400, "Shared secret registration is not enabled")
|
||||
|
||||
body = parse_json_object_from_request(request)
|
||||
|
||||
if "nonce" not in body:
|
||||
raise SynapseError(400, "nonce must be specified", errcode=Codes.BAD_JSON)
|
||||
|
||||
nonce = body["nonce"]
|
||||
|
||||
if nonce not in self.nonces:
|
||||
raise SynapseError(400, "unrecognised nonce")
|
||||
|
||||
# Delete the nonce, so it can't be reused, even if it's invalid
|
||||
del self.nonces[nonce]
|
||||
|
||||
if "username" not in body:
|
||||
raise SynapseError(
|
||||
400, "username must be specified", errcode=Codes.BAD_JSON
|
||||
)
|
||||
else:
|
||||
if (
|
||||
not isinstance(body["username"], text_type)
|
||||
or len(body["username"]) > 512
|
||||
):
|
||||
raise SynapseError(400, "Invalid username")
|
||||
|
||||
username = body["username"].encode("utf-8")
|
||||
if b"\x00" in username:
|
||||
raise SynapseError(400, "Invalid username")
|
||||
|
||||
if "password" not in body:
|
||||
raise SynapseError(
|
||||
400, "password must be specified", errcode=Codes.BAD_JSON
|
||||
)
|
||||
else:
|
||||
if (
|
||||
not isinstance(body["password"], text_type)
|
||||
or len(body["password"]) > 512
|
||||
):
|
||||
raise SynapseError(400, "Invalid password")
|
||||
|
||||
password = body["password"].encode("utf-8")
|
||||
if b"\x00" in password:
|
||||
raise SynapseError(400, "Invalid password")
|
||||
|
||||
admin = body.get("admin", None)
|
||||
user_type = body.get("user_type", None)
|
||||
|
||||
if user_type is not None and user_type not in UserTypes.ALL_USER_TYPES:
|
||||
raise SynapseError(400, "Invalid user type")
|
||||
|
||||
got_mac = body["mac"]
|
||||
|
||||
want_mac = hmac.new(
|
||||
key=self.hs.config.registration_shared_secret.encode(),
|
||||
digestmod=hashlib.sha1,
|
||||
)
|
||||
want_mac.update(nonce.encode("utf8"))
|
||||
want_mac.update(b"\x00")
|
||||
want_mac.update(username)
|
||||
want_mac.update(b"\x00")
|
||||
want_mac.update(password)
|
||||
want_mac.update(b"\x00")
|
||||
want_mac.update(b"admin" if admin else b"notadmin")
|
||||
if user_type:
|
||||
want_mac.update(b"\x00")
|
||||
want_mac.update(user_type.encode("utf8"))
|
||||
want_mac = want_mac.hexdigest()
|
||||
|
||||
if not hmac.compare_digest(want_mac.encode("ascii"), got_mac.encode("ascii")):
|
||||
raise SynapseError(403, "HMAC incorrect")
|
||||
|
||||
# Reuse the parts of RegisterRestServlet to reduce code duplication
|
||||
from synapse.rest.client.v2_alpha.register import RegisterRestServlet
|
||||
|
||||
register = RegisterRestServlet(self.hs)
|
||||
|
||||
user_id = await register.registration_handler.register_user(
|
||||
localpart=body["username"].lower(),
|
||||
password=body["password"],
|
||||
admin=bool(admin),
|
||||
user_type=user_type,
|
||||
)
|
||||
|
||||
result = await register._create_registration_details(user_id, body)
|
||||
return 200, result
|
||||
|
||||
|
||||
class WhoisRestServlet(RestServlet):
|
||||
PATTERNS = historical_admin_path_patterns("/whois/(?P<user_id>[^/]*)")
|
||||
|
||||
def __init__(self, hs):
|
||||
self.hs = hs
|
||||
self.auth = hs.get_auth()
|
||||
self.handlers = hs.get_handlers()
|
||||
|
||||
async def on_GET(self, request, user_id):
|
||||
target_user = UserID.from_string(user_id)
|
||||
requester = await self.auth.get_user_by_req(request)
|
||||
auth_user = requester.user
|
||||
|
||||
if target_user != auth_user:
|
||||
await assert_user_is_admin(self.auth, auth_user)
|
||||
|
||||
if not self.hs.is_mine(target_user):
|
||||
raise SynapseError(400, "Can only whois a local user")
|
||||
|
||||
ret = await self.handlers.admin_handler.get_whois(target_user)
|
||||
|
||||
return 200, ret
|
||||
|
||||
|
||||
class PurgeHistoryRestServlet(RestServlet):
|
||||
PATTERNS = historical_admin_path_patterns(
|
||||
"/purge_history/(?P<room_id>[^/]*)(/(?P<event_id>[^/]+))?"
|
||||
@@ -342,369 +166,6 @@ class PurgeHistoryStatusRestServlet(RestServlet):
|
||||
return 200, purge_status.asdict()
|
||||
|
||||
|
||||
class DeactivateAccountRestServlet(RestServlet):
|
||||
PATTERNS = historical_admin_path_patterns("/deactivate/(?P<target_user_id>[^/]*)")
|
||||
|
||||
def __init__(self, hs):
|
||||
self._deactivate_account_handler = hs.get_deactivate_account_handler()
|
||||
self.auth = hs.get_auth()
|
||||
|
||||
async def on_POST(self, request, target_user_id):
|
||||
await assert_requester_is_admin(self.auth, request)
|
||||
body = parse_json_object_from_request(request, allow_empty_body=True)
|
||||
erase = body.get("erase", False)
|
||||
if not isinstance(erase, bool):
|
||||
raise SynapseError(
|
||||
http_client.BAD_REQUEST,
|
||||
"Param 'erase' must be a boolean, if given",
|
||||
Codes.BAD_JSON,
|
||||
)
|
||||
|
||||
UserID.from_string(target_user_id)
|
||||
|
||||
result = await self._deactivate_account_handler.deactivate_account(
|
||||
target_user_id, erase
|
||||
)
|
||||
if result:
|
||||
id_server_unbind_result = "success"
|
||||
else:
|
||||
id_server_unbind_result = "no-support"
|
||||
|
||||
return 200, {"id_server_unbind_result": id_server_unbind_result}
|
||||
|
||||
|
||||
class ShutdownRoomRestServlet(RestServlet):
|
||||
"""Shuts down a room by removing all local users from the room and blocking
|
||||
all future invites and joins to the room. Any local aliases will be repointed
|
||||
to a new room created by `new_room_user_id` and kicked users will be auto
|
||||
joined to the new room.
|
||||
"""
|
||||
|
||||
PATTERNS = historical_admin_path_patterns("/shutdown_room/(?P<room_id>[^/]+)")
|
||||
|
||||
DEFAULT_MESSAGE = (
|
||||
"Sharing illegal content on this server is not permitted and rooms in"
|
||||
" violation will be blocked."
|
||||
)
|
||||
|
||||
def __init__(self, hs):
|
||||
self.hs = hs
|
||||
self.store = hs.get_datastore()
|
||||
self.state = hs.get_state_handler()
|
||||
self._room_creation_handler = hs.get_room_creation_handler()
|
||||
self.event_creation_handler = hs.get_event_creation_handler()
|
||||
self.room_member_handler = hs.get_room_member_handler()
|
||||
self.auth = hs.get_auth()
|
||||
|
||||
async def on_POST(self, request, room_id):
|
||||
requester = await self.auth.get_user_by_req(request)
|
||||
await assert_user_is_admin(self.auth, requester.user)
|
||||
|
||||
content = parse_json_object_from_request(request)
|
||||
assert_params_in_dict(content, ["new_room_user_id"])
|
||||
new_room_user_id = content["new_room_user_id"]
|
||||
|
||||
room_creator_requester = create_requester(new_room_user_id)
|
||||
|
||||
message = content.get("message", self.DEFAULT_MESSAGE)
|
||||
room_name = content.get("room_name", "Content Violation Notification")
|
||||
|
||||
info = await self._room_creation_handler.create_room(
|
||||
room_creator_requester,
|
||||
config={
|
||||
"preset": "public_chat",
|
||||
"name": room_name,
|
||||
"power_level_content_override": {"users_default": -10},
|
||||
},
|
||||
ratelimit=False,
|
||||
)
|
||||
new_room_id = info["room_id"]
|
||||
|
||||
requester_user_id = requester.user.to_string()
|
||||
|
||||
logger.info(
|
||||
"Shutting down room %r, joining to new room: %r", room_id, new_room_id
|
||||
)
|
||||
|
||||
# This will work even if the room is already blocked, but that is
|
||||
# desirable in case the first attempt at blocking the room failed below.
|
||||
await self.store.block_room(room_id, requester_user_id)
|
||||
|
||||
users = await self.state.get_current_users_in_room(room_id)
|
||||
kicked_users = []
|
||||
failed_to_kick_users = []
|
||||
for user_id in users:
|
||||
if not self.hs.is_mine_id(user_id):
|
||||
continue
|
||||
|
||||
logger.info("Kicking %r from %r...", user_id, room_id)
|
||||
|
||||
try:
|
||||
target_requester = create_requester(user_id)
|
||||
await self.room_member_handler.update_membership(
|
||||
requester=target_requester,
|
||||
target=target_requester.user,
|
||||
room_id=room_id,
|
||||
action=Membership.LEAVE,
|
||||
content={},
|
||||
ratelimit=False,
|
||||
require_consent=False,
|
||||
)
|
||||
|
||||
await self.room_member_handler.forget(target_requester.user, room_id)
|
||||
|
||||
await self.room_member_handler.update_membership(
|
||||
requester=target_requester,
|
||||
target=target_requester.user,
|
||||
room_id=new_room_id,
|
||||
action=Membership.JOIN,
|
||||
content={},
|
||||
ratelimit=False,
|
||||
require_consent=False,
|
||||
)
|
||||
|
||||
kicked_users.append(user_id)
|
||||
except Exception:
|
||||
logger.exception(
|
||||
"Failed to leave old room and join new room for %r", user_id
|
||||
)
|
||||
failed_to_kick_users.append(user_id)
|
||||
|
||||
await self.event_creation_handler.create_and_send_nonmember_event(
|
||||
room_creator_requester,
|
||||
{
|
||||
"type": "m.room.message",
|
||||
"content": {"body": message, "msgtype": "m.text"},
|
||||
"room_id": new_room_id,
|
||||
"sender": new_room_user_id,
|
||||
},
|
||||
ratelimit=False,
|
||||
)
|
||||
|
||||
aliases_for_room = await maybe_awaitable(
|
||||
self.store.get_aliases_for_room(room_id)
|
||||
)
|
||||
|
||||
await self.store.update_aliases_for_room(
|
||||
room_id, new_room_id, requester_user_id
|
||||
)
|
||||
|
||||
return (
|
||||
200,
|
||||
{
|
||||
"kicked_users": kicked_users,
|
||||
"failed_to_kick_users": failed_to_kick_users,
|
||||
"local_aliases": aliases_for_room,
|
||||
"new_room_id": new_room_id,
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
class ResetPasswordRestServlet(RestServlet):
|
||||
"""Post request to allow an administrator reset password for a user.
|
||||
This needs user to have administrator access in Synapse.
|
||||
Example:
|
||||
http://localhost:8008/_synapse/admin/v1/reset_password/
|
||||
@user:to_reset_password?access_token=admin_access_token
|
||||
JsonBodyToSend:
|
||||
{
|
||||
"new_password": "secret"
|
||||
}
|
||||
Returns:
|
||||
200 OK with empty object if success otherwise an error.
|
||||
"""
|
||||
|
||||
PATTERNS = historical_admin_path_patterns(
|
||||
"/reset_password/(?P<target_user_id>[^/]*)"
|
||||
)
|
||||
|
||||
def __init__(self, hs):
|
||||
self.store = hs.get_datastore()
|
||||
self.hs = hs
|
||||
self.auth = hs.get_auth()
|
||||
self._set_password_handler = hs.get_set_password_handler()
|
||||
|
||||
async def on_POST(self, request, target_user_id):
|
||||
"""Post request to allow an administrator reset password for a user.
|
||||
This needs user to have administrator access in Synapse.
|
||||
"""
|
||||
requester = await self.auth.get_user_by_req(request)
|
||||
await assert_user_is_admin(self.auth, requester.user)
|
||||
|
||||
UserID.from_string(target_user_id)
|
||||
|
||||
params = parse_json_object_from_request(request)
|
||||
assert_params_in_dict(params, ["new_password"])
|
||||
new_password = params["new_password"]
|
||||
|
||||
await self._set_password_handler.set_password(
|
||||
target_user_id, new_password, requester
|
||||
)
|
||||
return 200, {}
|
||||
|
||||
|
||||
class GetUsersPaginatedRestServlet(RestServlet):
|
||||
"""Get request to get specific number of users from Synapse.
|
||||
This needs user to have administrator access in Synapse.
|
||||
Example:
|
||||
http://localhost:8008/_synapse/admin/v1/users_paginate/
|
||||
@admin:user?access_token=admin_access_token&start=0&limit=10
|
||||
Returns:
|
||||
200 OK with json object {list[dict[str, Any]], count} or empty object.
|
||||
"""
|
||||
|
||||
PATTERNS = historical_admin_path_patterns(
|
||||
"/users_paginate/(?P<target_user_id>[^/]*)"
|
||||
)
|
||||
|
||||
def __init__(self, hs):
|
||||
self.store = hs.get_datastore()
|
||||
self.hs = hs
|
||||
self.auth = hs.get_auth()
|
||||
self.handlers = hs.get_handlers()
|
||||
|
||||
async def on_GET(self, request, target_user_id):
|
||||
"""Get request to get specific number of users from Synapse.
|
||||
This needs user to have administrator access in Synapse.
|
||||
"""
|
||||
await assert_requester_is_admin(self.auth, request)
|
||||
|
||||
target_user = UserID.from_string(target_user_id)
|
||||
|
||||
if not self.hs.is_mine(target_user):
|
||||
raise SynapseError(400, "Can only users a local user")
|
||||
|
||||
order = "name" # order by name in user table
|
||||
start = parse_integer(request, "start", required=True)
|
||||
limit = parse_integer(request, "limit", required=True)
|
||||
|
||||
logger.info("limit: %s, start: %s", limit, start)
|
||||
|
||||
ret = await self.handlers.admin_handler.get_users_paginate(order, start, limit)
|
||||
return 200, ret
|
||||
|
||||
async def on_POST(self, request, target_user_id):
|
||||
"""Post request to get specific number of users from Synapse..
|
||||
This needs user to have administrator access in Synapse.
|
||||
Example:
|
||||
http://localhost:8008/_synapse/admin/v1/users_paginate/
|
||||
@admin:user?access_token=admin_access_token
|
||||
JsonBodyToSend:
|
||||
{
|
||||
"start": "0",
|
||||
"limit": "10
|
||||
}
|
||||
Returns:
|
||||
200 OK with json object {list[dict[str, Any]], count} or empty object.
|
||||
"""
|
||||
await assert_requester_is_admin(self.auth, request)
|
||||
UserID.from_string(target_user_id)
|
||||
|
||||
order = "name" # order by name in user table
|
||||
params = parse_json_object_from_request(request)
|
||||
assert_params_in_dict(params, ["limit", "start"])
|
||||
limit = params["limit"]
|
||||
start = params["start"]
|
||||
logger.info("limit: %s, start: %s", limit, start)
|
||||
|
||||
ret = await self.handlers.admin_handler.get_users_paginate(order, start, limit)
|
||||
return 200, ret
|
||||
|
||||
|
||||
class SearchUsersRestServlet(RestServlet):
|
||||
"""Get request to search user table for specific users according to
|
||||
search term.
|
||||
This needs user to have administrator access in Synapse.
|
||||
Example:
|
||||
http://localhost:8008/_synapse/admin/v1/search_users/
|
||||
@admin:user?access_token=admin_access_token&term=alice
|
||||
Returns:
|
||||
200 OK with json object {list[dict[str, Any]], count} or empty object.
|
||||
"""
|
||||
|
||||
PATTERNS = historical_admin_path_patterns("/search_users/(?P<target_user_id>[^/]*)")
|
||||
|
||||
def __init__(self, hs):
|
||||
self.store = hs.get_datastore()
|
||||
self.hs = hs
|
||||
self.auth = hs.get_auth()
|
||||
self.handlers = hs.get_handlers()
|
||||
|
||||
async def on_GET(self, request, target_user_id):
|
||||
"""Get request to search user table for specific users according to
|
||||
search term.
|
||||
This needs user to have a administrator access in Synapse.
|
||||
"""
|
||||
await assert_requester_is_admin(self.auth, request)
|
||||
|
||||
target_user = UserID.from_string(target_user_id)
|
||||
|
||||
# To allow all users to get the users list
|
||||
# if not is_admin and target_user != auth_user:
|
||||
# raise AuthError(403, "You are not a server admin")
|
||||
|
||||
if not self.hs.is_mine(target_user):
|
||||
raise SynapseError(400, "Can only users a local user")
|
||||
|
||||
term = parse_string(request, "term", required=True)
|
||||
logger.info("term: %s ", term)
|
||||
|
||||
ret = await self.handlers.admin_handler.search_users(term)
|
||||
return 200, ret
|
||||
|
||||
|
||||
class DeleteGroupAdminRestServlet(RestServlet):
|
||||
"""Allows deleting of local groups
|
||||
"""
|
||||
|
||||
PATTERNS = historical_admin_path_patterns("/delete_group/(?P<group_id>[^/]*)")
|
||||
|
||||
def __init__(self, hs):
|
||||
self.group_server = hs.get_groups_server_handler()
|
||||
self.is_mine_id = hs.is_mine_id
|
||||
self.auth = hs.get_auth()
|
||||
|
||||
async def on_POST(self, request, group_id):
|
||||
requester = await self.auth.get_user_by_req(request)
|
||||
await assert_user_is_admin(self.auth, requester.user)
|
||||
|
||||
if not self.is_mine_id(group_id):
|
||||
raise SynapseError(400, "Can only delete local groups")
|
||||
|
||||
await self.group_server.delete_group(group_id, requester.user.to_string())
|
||||
return 200, {}
|
||||
|
||||
|
||||
class AccountValidityRenewServlet(RestServlet):
|
||||
PATTERNS = historical_admin_path_patterns("/account_validity/validity$")
|
||||
|
||||
def __init__(self, hs):
|
||||
"""
|
||||
Args:
|
||||
hs (synapse.server.HomeServer): server
|
||||
"""
|
||||
self.hs = hs
|
||||
self.account_activity_handler = hs.get_account_validity_handler()
|
||||
self.auth = hs.get_auth()
|
||||
|
||||
async def on_POST(self, request):
|
||||
await assert_requester_is_admin(self.auth, request)
|
||||
|
||||
body = parse_json_object_from_request(request)
|
||||
|
||||
if "user_id" not in body:
|
||||
raise SynapseError(400, "Missing property 'user_id' in the request body")
|
||||
|
||||
expiration_ts = await self.account_activity_handler.renew_account_for_user(
|
||||
body["user_id"],
|
||||
body.get("expiration_ts"),
|
||||
not body.get("enable_renewal_emails", True),
|
||||
)
|
||||
|
||||
res = {"expiration_ts": expiration_ts}
|
||||
return 200, res
|
||||
|
||||
|
||||
########################################################################################
|
||||
#
|
||||
# please don't add more servlets here: this file is already long and unwieldy. Put
|
||||
|
||||
46
synapse/rest/admin/groups.py
Normal file
46
synapse/rest/admin/groups.py
Normal file
@@ -0,0 +1,46 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
# Copyright 2019 The Matrix.org Foundation C.I.C.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
import logging
|
||||
|
||||
from synapse.api.errors import SynapseError
|
||||
from synapse.http.servlet import RestServlet
|
||||
from synapse.rest.admin._base import (
|
||||
assert_user_is_admin,
|
||||
historical_admin_path_patterns,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class DeleteGroupAdminRestServlet(RestServlet):
|
||||
"""Allows deleting of local groups
|
||||
"""
|
||||
|
||||
PATTERNS = historical_admin_path_patterns("/delete_group/(?P<group_id>[^/]*)")
|
||||
|
||||
def __init__(self, hs):
|
||||
self.group_server = hs.get_groups_server_handler()
|
||||
self.is_mine_id = hs.is_mine_id
|
||||
self.auth = hs.get_auth()
|
||||
|
||||
async def on_POST(self, request, group_id):
|
||||
requester = await self.auth.get_user_by_req(request)
|
||||
await assert_user_is_admin(self.auth, requester.user)
|
||||
|
||||
if not self.is_mine_id(group_id):
|
||||
raise SynapseError(400, "Can only delete local groups")
|
||||
|
||||
await self.group_server.delete_group(group_id, requester.user.to_string())
|
||||
return 200, {}
|
||||
157
synapse/rest/admin/rooms.py
Normal file
157
synapse/rest/admin/rooms.py
Normal file
@@ -0,0 +1,157 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
# Copyright 2019 The Matrix.org Foundation C.I.C.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
import logging
|
||||
|
||||
from synapse.api.constants import Membership
|
||||
from synapse.http.servlet import (
|
||||
RestServlet,
|
||||
assert_params_in_dict,
|
||||
parse_json_object_from_request,
|
||||
)
|
||||
from synapse.rest.admin._base import (
|
||||
assert_user_is_admin,
|
||||
historical_admin_path_patterns,
|
||||
)
|
||||
from synapse.types import create_requester
|
||||
from synapse.util.async_helpers import maybe_awaitable
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class ShutdownRoomRestServlet(RestServlet):
|
||||
"""Shuts down a room by removing all local users from the room and blocking
|
||||
all future invites and joins to the room. Any local aliases will be repointed
|
||||
to a new room created by `new_room_user_id` and kicked users will be auto
|
||||
joined to the new room.
|
||||
"""
|
||||
|
||||
PATTERNS = historical_admin_path_patterns("/shutdown_room/(?P<room_id>[^/]+)")
|
||||
|
||||
DEFAULT_MESSAGE = (
|
||||
"Sharing illegal content on this server is not permitted and rooms in"
|
||||
" violation will be blocked."
|
||||
)
|
||||
|
||||
def __init__(self, hs):
|
||||
self.hs = hs
|
||||
self.store = hs.get_datastore()
|
||||
self.state = hs.get_state_handler()
|
||||
self._room_creation_handler = hs.get_room_creation_handler()
|
||||
self.event_creation_handler = hs.get_event_creation_handler()
|
||||
self.room_member_handler = hs.get_room_member_handler()
|
||||
self.auth = hs.get_auth()
|
||||
|
||||
async def on_POST(self, request, room_id):
|
||||
requester = await self.auth.get_user_by_req(request)
|
||||
await assert_user_is_admin(self.auth, requester.user)
|
||||
|
||||
content = parse_json_object_from_request(request)
|
||||
assert_params_in_dict(content, ["new_room_user_id"])
|
||||
new_room_user_id = content["new_room_user_id"]
|
||||
|
||||
room_creator_requester = create_requester(new_room_user_id)
|
||||
|
||||
message = content.get("message", self.DEFAULT_MESSAGE)
|
||||
room_name = content.get("room_name", "Content Violation Notification")
|
||||
|
||||
info = await self._room_creation_handler.create_room(
|
||||
room_creator_requester,
|
||||
config={
|
||||
"preset": "public_chat",
|
||||
"name": room_name,
|
||||
"power_level_content_override": {"users_default": -10},
|
||||
},
|
||||
ratelimit=False,
|
||||
)
|
||||
new_room_id = info["room_id"]
|
||||
|
||||
requester_user_id = requester.user.to_string()
|
||||
|
||||
logger.info(
|
||||
"Shutting down room %r, joining to new room: %r", room_id, new_room_id
|
||||
)
|
||||
|
||||
# This will work even if the room is already blocked, but that is
|
||||
# desirable in case the first attempt at blocking the room failed below.
|
||||
await self.store.block_room(room_id, requester_user_id)
|
||||
|
||||
users = await self.state.get_current_users_in_room(room_id)
|
||||
kicked_users = []
|
||||
failed_to_kick_users = []
|
||||
for user_id in users:
|
||||
if not self.hs.is_mine_id(user_id):
|
||||
continue
|
||||
|
||||
logger.info("Kicking %r from %r...", user_id, room_id)
|
||||
|
||||
try:
|
||||
target_requester = create_requester(user_id)
|
||||
await self.room_member_handler.update_membership(
|
||||
requester=target_requester,
|
||||
target=target_requester.user,
|
||||
room_id=room_id,
|
||||
action=Membership.LEAVE,
|
||||
content={},
|
||||
ratelimit=False,
|
||||
require_consent=False,
|
||||
)
|
||||
|
||||
await self.room_member_handler.forget(target_requester.user, room_id)
|
||||
|
||||
await self.room_member_handler.update_membership(
|
||||
requester=target_requester,
|
||||
target=target_requester.user,
|
||||
room_id=new_room_id,
|
||||
action=Membership.JOIN,
|
||||
content={},
|
||||
ratelimit=False,
|
||||
require_consent=False,
|
||||
)
|
||||
|
||||
kicked_users.append(user_id)
|
||||
except Exception:
|
||||
logger.exception(
|
||||
"Failed to leave old room and join new room for %r", user_id
|
||||
)
|
||||
failed_to_kick_users.append(user_id)
|
||||
|
||||
await self.event_creation_handler.create_and_send_nonmember_event(
|
||||
room_creator_requester,
|
||||
{
|
||||
"type": "m.room.message",
|
||||
"content": {"body": message, "msgtype": "m.text"},
|
||||
"room_id": new_room_id,
|
||||
"sender": new_room_user_id,
|
||||
},
|
||||
ratelimit=False,
|
||||
)
|
||||
|
||||
aliases_for_room = await maybe_awaitable(
|
||||
self.store.get_aliases_for_room(room_id)
|
||||
)
|
||||
|
||||
await self.store.update_aliases_for_room(
|
||||
room_id, new_room_id, requester_user_id
|
||||
)
|
||||
|
||||
return (
|
||||
200,
|
||||
{
|
||||
"kicked_users": kicked_users,
|
||||
"failed_to_kick_users": failed_to_kick_users,
|
||||
"local_aliases": aliases_for_room,
|
||||
"new_room_id": new_room_id,
|
||||
},
|
||||
)
|
||||
@@ -12,17 +12,419 @@
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
import hashlib
|
||||
import hmac
|
||||
import logging
|
||||
import re
|
||||
|
||||
from synapse.api.errors import SynapseError
|
||||
from six import text_type
|
||||
from six.moves import http_client
|
||||
|
||||
from synapse.api.constants import UserTypes
|
||||
from synapse.api.errors import Codes, SynapseError
|
||||
from synapse.http.servlet import (
|
||||
RestServlet,
|
||||
assert_params_in_dict,
|
||||
parse_integer,
|
||||
parse_json_object_from_request,
|
||||
parse_string,
|
||||
)
|
||||
from synapse.rest.admin._base import (
|
||||
assert_requester_is_admin,
|
||||
assert_user_is_admin,
|
||||
historical_admin_path_patterns,
|
||||
)
|
||||
from synapse.rest.admin import assert_requester_is_admin, assert_user_is_admin
|
||||
from synapse.types import UserID
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class UsersRestServlet(RestServlet):
|
||||
PATTERNS = historical_admin_path_patterns("/users/(?P<user_id>[^/]*)$")
|
||||
|
||||
def __init__(self, hs):
|
||||
self.hs = hs
|
||||
self.auth = hs.get_auth()
|
||||
self.admin_handler = hs.get_handlers().admin_handler
|
||||
|
||||
async def on_GET(self, request, user_id):
|
||||
target_user = UserID.from_string(user_id)
|
||||
await assert_requester_is_admin(self.auth, request)
|
||||
|
||||
if not self.hs.is_mine(target_user):
|
||||
raise SynapseError(400, "Can only users a local user")
|
||||
|
||||
ret = await self.admin_handler.get_users()
|
||||
|
||||
return 200, ret
|
||||
|
||||
|
||||
class GetUsersPaginatedRestServlet(RestServlet):
|
||||
"""Get request to get specific number of users from Synapse.
|
||||
This needs user to have administrator access in Synapse.
|
||||
Example:
|
||||
http://localhost:8008/_synapse/admin/v1/users_paginate/
|
||||
@admin:user?access_token=admin_access_token&start=0&limit=10
|
||||
Returns:
|
||||
200 OK with json object {list[dict[str, Any]], count} or empty object.
|
||||
"""
|
||||
|
||||
PATTERNS = historical_admin_path_patterns(
|
||||
"/users_paginate/(?P<target_user_id>[^/]*)"
|
||||
)
|
||||
|
||||
def __init__(self, hs):
|
||||
self.store = hs.get_datastore()
|
||||
self.hs = hs
|
||||
self.auth = hs.get_auth()
|
||||
self.handlers = hs.get_handlers()
|
||||
|
||||
async def on_GET(self, request, target_user_id):
|
||||
"""Get request to get specific number of users from Synapse.
|
||||
This needs user to have administrator access in Synapse.
|
||||
"""
|
||||
await assert_requester_is_admin(self.auth, request)
|
||||
|
||||
target_user = UserID.from_string(target_user_id)
|
||||
|
||||
if not self.hs.is_mine(target_user):
|
||||
raise SynapseError(400, "Can only users a local user")
|
||||
|
||||
order = "name" # order by name in user table
|
||||
start = parse_integer(request, "start", required=True)
|
||||
limit = parse_integer(request, "limit", required=True)
|
||||
|
||||
logger.info("limit: %s, start: %s", limit, start)
|
||||
|
||||
ret = await self.handlers.admin_handler.get_users_paginate(order, start, limit)
|
||||
return 200, ret
|
||||
|
||||
async def on_POST(self, request, target_user_id):
|
||||
"""Post request to get specific number of users from Synapse..
|
||||
This needs user to have administrator access in Synapse.
|
||||
Example:
|
||||
http://localhost:8008/_synapse/admin/v1/users_paginate/
|
||||
@admin:user?access_token=admin_access_token
|
||||
JsonBodyToSend:
|
||||
{
|
||||
"start": "0",
|
||||
"limit": "10
|
||||
}
|
||||
Returns:
|
||||
200 OK with json object {list[dict[str, Any]], count} or empty object.
|
||||
"""
|
||||
await assert_requester_is_admin(self.auth, request)
|
||||
UserID.from_string(target_user_id)
|
||||
|
||||
order = "name" # order by name in user table
|
||||
params = parse_json_object_from_request(request)
|
||||
assert_params_in_dict(params, ["limit", "start"])
|
||||
limit = params["limit"]
|
||||
start = params["start"]
|
||||
logger.info("limit: %s, start: %s", limit, start)
|
||||
|
||||
ret = await self.handlers.admin_handler.get_users_paginate(order, start, limit)
|
||||
return 200, ret
|
||||
|
||||
|
||||
class UserRegisterServlet(RestServlet):
|
||||
"""
|
||||
Attributes:
|
||||
NONCE_TIMEOUT (int): Seconds until a generated nonce won't be accepted
|
||||
nonces (dict[str, int]): The nonces that we will accept. A dict of
|
||||
nonce to the time it was generated, in int seconds.
|
||||
"""
|
||||
|
||||
PATTERNS = historical_admin_path_patterns("/register")
|
||||
NONCE_TIMEOUT = 60
|
||||
|
||||
def __init__(self, hs):
|
||||
self.handlers = hs.get_handlers()
|
||||
self.reactor = hs.get_reactor()
|
||||
self.nonces = {}
|
||||
self.hs = hs
|
||||
|
||||
def _clear_old_nonces(self):
|
||||
"""
|
||||
Clear out old nonces that are older than NONCE_TIMEOUT.
|
||||
"""
|
||||
now = int(self.reactor.seconds())
|
||||
|
||||
for k, v in list(self.nonces.items()):
|
||||
if now - v > self.NONCE_TIMEOUT:
|
||||
del self.nonces[k]
|
||||
|
||||
def on_GET(self, request):
|
||||
"""
|
||||
Generate a new nonce.
|
||||
"""
|
||||
self._clear_old_nonces()
|
||||
|
||||
nonce = self.hs.get_secrets().token_hex(64)
|
||||
self.nonces[nonce] = int(self.reactor.seconds())
|
||||
return 200, {"nonce": nonce}
|
||||
|
||||
async def on_POST(self, request):
|
||||
self._clear_old_nonces()
|
||||
|
||||
if not self.hs.config.registration_shared_secret:
|
||||
raise SynapseError(400, "Shared secret registration is not enabled")
|
||||
|
||||
body = parse_json_object_from_request(request)
|
||||
|
||||
if "nonce" not in body:
|
||||
raise SynapseError(400, "nonce must be specified", errcode=Codes.BAD_JSON)
|
||||
|
||||
nonce = body["nonce"]
|
||||
|
||||
if nonce not in self.nonces:
|
||||
raise SynapseError(400, "unrecognised nonce")
|
||||
|
||||
# Delete the nonce, so it can't be reused, even if it's invalid
|
||||
del self.nonces[nonce]
|
||||
|
||||
if "username" not in body:
|
||||
raise SynapseError(
|
||||
400, "username must be specified", errcode=Codes.BAD_JSON
|
||||
)
|
||||
else:
|
||||
if (
|
||||
not isinstance(body["username"], text_type)
|
||||
or len(body["username"]) > 512
|
||||
):
|
||||
raise SynapseError(400, "Invalid username")
|
||||
|
||||
username = body["username"].encode("utf-8")
|
||||
if b"\x00" in username:
|
||||
raise SynapseError(400, "Invalid username")
|
||||
|
||||
if "password" not in body:
|
||||
raise SynapseError(
|
||||
400, "password must be specified", errcode=Codes.BAD_JSON
|
||||
)
|
||||
else:
|
||||
if (
|
||||
not isinstance(body["password"], text_type)
|
||||
or len(body["password"]) > 512
|
||||
):
|
||||
raise SynapseError(400, "Invalid password")
|
||||
|
||||
password = body["password"].encode("utf-8")
|
||||
if b"\x00" in password:
|
||||
raise SynapseError(400, "Invalid password")
|
||||
|
||||
admin = body.get("admin", None)
|
||||
user_type = body.get("user_type", None)
|
||||
|
||||
if user_type is not None and user_type not in UserTypes.ALL_USER_TYPES:
|
||||
raise SynapseError(400, "Invalid user type")
|
||||
|
||||
got_mac = body["mac"]
|
||||
|
||||
want_mac = hmac.new(
|
||||
key=self.hs.config.registration_shared_secret.encode(),
|
||||
digestmod=hashlib.sha1,
|
||||
)
|
||||
want_mac.update(nonce.encode("utf8"))
|
||||
want_mac.update(b"\x00")
|
||||
want_mac.update(username)
|
||||
want_mac.update(b"\x00")
|
||||
want_mac.update(password)
|
||||
want_mac.update(b"\x00")
|
||||
want_mac.update(b"admin" if admin else b"notadmin")
|
||||
if user_type:
|
||||
want_mac.update(b"\x00")
|
||||
want_mac.update(user_type.encode("utf8"))
|
||||
want_mac = want_mac.hexdigest()
|
||||
|
||||
if not hmac.compare_digest(want_mac.encode("ascii"), got_mac.encode("ascii")):
|
||||
raise SynapseError(403, "HMAC incorrect")
|
||||
|
||||
# Reuse the parts of RegisterRestServlet to reduce code duplication
|
||||
from synapse.rest.client.v2_alpha.register import RegisterRestServlet
|
||||
|
||||
register = RegisterRestServlet(self.hs)
|
||||
|
||||
user_id = await register.registration_handler.register_user(
|
||||
localpart=body["username"].lower(),
|
||||
password=body["password"],
|
||||
admin=bool(admin),
|
||||
user_type=user_type,
|
||||
)
|
||||
|
||||
result = await register._create_registration_details(user_id, body)
|
||||
return 200, result
|
||||
|
||||
|
||||
class WhoisRestServlet(RestServlet):
|
||||
PATTERNS = historical_admin_path_patterns("/whois/(?P<user_id>[^/]*)")
|
||||
|
||||
def __init__(self, hs):
|
||||
self.hs = hs
|
||||
self.auth = hs.get_auth()
|
||||
self.handlers = hs.get_handlers()
|
||||
|
||||
async def on_GET(self, request, user_id):
|
||||
target_user = UserID.from_string(user_id)
|
||||
requester = await self.auth.get_user_by_req(request)
|
||||
auth_user = requester.user
|
||||
|
||||
if target_user != auth_user:
|
||||
await assert_user_is_admin(self.auth, auth_user)
|
||||
|
||||
if not self.hs.is_mine(target_user):
|
||||
raise SynapseError(400, "Can only whois a local user")
|
||||
|
||||
ret = await self.handlers.admin_handler.get_whois(target_user)
|
||||
|
||||
return 200, ret
|
||||
|
||||
|
||||
class DeactivateAccountRestServlet(RestServlet):
|
||||
PATTERNS = historical_admin_path_patterns("/deactivate/(?P<target_user_id>[^/]*)")
|
||||
|
||||
def __init__(self, hs):
|
||||
self._deactivate_account_handler = hs.get_deactivate_account_handler()
|
||||
self.auth = hs.get_auth()
|
||||
|
||||
async def on_POST(self, request, target_user_id):
|
||||
await assert_requester_is_admin(self.auth, request)
|
||||
body = parse_json_object_from_request(request, allow_empty_body=True)
|
||||
erase = body.get("erase", False)
|
||||
if not isinstance(erase, bool):
|
||||
raise SynapseError(
|
||||
http_client.BAD_REQUEST,
|
||||
"Param 'erase' must be a boolean, if given",
|
||||
Codes.BAD_JSON,
|
||||
)
|
||||
|
||||
UserID.from_string(target_user_id)
|
||||
|
||||
result = await self._deactivate_account_handler.deactivate_account(
|
||||
target_user_id, erase
|
||||
)
|
||||
if result:
|
||||
id_server_unbind_result = "success"
|
||||
else:
|
||||
id_server_unbind_result = "no-support"
|
||||
|
||||
return 200, {"id_server_unbind_result": id_server_unbind_result}
|
||||
|
||||
|
||||
class AccountValidityRenewServlet(RestServlet):
|
||||
PATTERNS = historical_admin_path_patterns("/account_validity/validity$")
|
||||
|
||||
def __init__(self, hs):
|
||||
"""
|
||||
Args:
|
||||
hs (synapse.server.HomeServer): server
|
||||
"""
|
||||
self.hs = hs
|
||||
self.account_activity_handler = hs.get_account_validity_handler()
|
||||
self.auth = hs.get_auth()
|
||||
|
||||
async def on_POST(self, request):
|
||||
await assert_requester_is_admin(self.auth, request)
|
||||
|
||||
body = parse_json_object_from_request(request)
|
||||
|
||||
if "user_id" not in body:
|
||||
raise SynapseError(400, "Missing property 'user_id' in the request body")
|
||||
|
||||
expiration_ts = await self.account_activity_handler.renew_account_for_user(
|
||||
body["user_id"],
|
||||
body.get("expiration_ts"),
|
||||
not body.get("enable_renewal_emails", True),
|
||||
)
|
||||
|
||||
res = {"expiration_ts": expiration_ts}
|
||||
return 200, res
|
||||
|
||||
|
||||
class ResetPasswordRestServlet(RestServlet):
|
||||
"""Post request to allow an administrator reset password for a user.
|
||||
This needs user to have administrator access in Synapse.
|
||||
Example:
|
||||
http://localhost:8008/_synapse/admin/v1/reset_password/
|
||||
@user:to_reset_password?access_token=admin_access_token
|
||||
JsonBodyToSend:
|
||||
{
|
||||
"new_password": "secret"
|
||||
}
|
||||
Returns:
|
||||
200 OK with empty object if success otherwise an error.
|
||||
"""
|
||||
|
||||
PATTERNS = historical_admin_path_patterns(
|
||||
"/reset_password/(?P<target_user_id>[^/]*)"
|
||||
)
|
||||
|
||||
def __init__(self, hs):
|
||||
self.store = hs.get_datastore()
|
||||
self.hs = hs
|
||||
self.auth = hs.get_auth()
|
||||
self._set_password_handler = hs.get_set_password_handler()
|
||||
|
||||
async def on_POST(self, request, target_user_id):
|
||||
"""Post request to allow an administrator reset password for a user.
|
||||
This needs user to have administrator access in Synapse.
|
||||
"""
|
||||
requester = await self.auth.get_user_by_req(request)
|
||||
await assert_user_is_admin(self.auth, requester.user)
|
||||
|
||||
UserID.from_string(target_user_id)
|
||||
|
||||
params = parse_json_object_from_request(request)
|
||||
assert_params_in_dict(params, ["new_password"])
|
||||
new_password = params["new_password"]
|
||||
|
||||
await self._set_password_handler.set_password(
|
||||
target_user_id, new_password, requester
|
||||
)
|
||||
return 200, {}
|
||||
|
||||
|
||||
class SearchUsersRestServlet(RestServlet):
|
||||
"""Get request to search user table for specific users according to
|
||||
search term.
|
||||
This needs user to have administrator access in Synapse.
|
||||
Example:
|
||||
http://localhost:8008/_synapse/admin/v1/search_users/
|
||||
@admin:user?access_token=admin_access_token&term=alice
|
||||
Returns:
|
||||
200 OK with json object {list[dict[str, Any]], count} or empty object.
|
||||
"""
|
||||
|
||||
PATTERNS = historical_admin_path_patterns("/search_users/(?P<target_user_id>[^/]*)")
|
||||
|
||||
def __init__(self, hs):
|
||||
self.store = hs.get_datastore()
|
||||
self.hs = hs
|
||||
self.auth = hs.get_auth()
|
||||
self.handlers = hs.get_handlers()
|
||||
|
||||
async def on_GET(self, request, target_user_id):
|
||||
"""Get request to search user table for specific users according to
|
||||
search term.
|
||||
This needs user to have a administrator access in Synapse.
|
||||
"""
|
||||
await assert_requester_is_admin(self.auth, request)
|
||||
|
||||
target_user = UserID.from_string(target_user_id)
|
||||
|
||||
# To allow all users to get the users list
|
||||
# if not is_admin and target_user != auth_user:
|
||||
# raise AuthError(403, "You are not a server admin")
|
||||
|
||||
if not self.hs.is_mine(target_user):
|
||||
raise SynapseError(400, "Can only users a local user")
|
||||
|
||||
term = parse_string(request, "term", required=True)
|
||||
logger.info("term: %s ", term)
|
||||
|
||||
ret = await self.handlers.admin_handler.search_users(term)
|
||||
return 200, ret
|
||||
|
||||
|
||||
class UserAdminServlet(RestServlet):
|
||||
"""
|
||||
|
||||
@@ -92,8 +92,11 @@ class LoginRestServlet(RestServlet):
|
||||
self.auth_handler = self.hs.get_auth_handler()
|
||||
self.registration_handler = hs.get_registration_handler()
|
||||
self.handlers = hs.get_handlers()
|
||||
self._clock = hs.get_clock()
|
||||
self._well_known_builder = WellKnownBuilder(hs)
|
||||
self._address_ratelimiter = Ratelimiter()
|
||||
self._account_ratelimiter = Ratelimiter()
|
||||
self._failed_attempts_ratelimiter = Ratelimiter()
|
||||
|
||||
def on_GET(self, request):
|
||||
flows = []
|
||||
@@ -202,6 +205,16 @@ class LoginRestServlet(RestServlet):
|
||||
# (See add_threepid in synapse/handlers/auth.py)
|
||||
address = address.lower()
|
||||
|
||||
# We also apply account rate limiting using the 3PID as a key, as
|
||||
# otherwise using 3PID bypasses the ratelimiting based on user ID.
|
||||
self._failed_attempts_ratelimiter.ratelimit(
|
||||
(medium, address),
|
||||
time_now_s=self._clock.time(),
|
||||
rate_hz=self.hs.config.rc_login_failed_attempts.per_second,
|
||||
burst_count=self.hs.config.rc_login_failed_attempts.burst_count,
|
||||
update=False,
|
||||
)
|
||||
|
||||
# Check for login providers that support 3pid login types
|
||||
(
|
||||
canonical_user_id,
|
||||
@@ -211,7 +224,8 @@ class LoginRestServlet(RestServlet):
|
||||
)
|
||||
if canonical_user_id:
|
||||
# Authentication through password provider and 3pid succeeded
|
||||
result = yield self._register_device_with_callback(
|
||||
|
||||
result = yield self._complete_login(
|
||||
canonical_user_id, login_submission, callback_3pid
|
||||
)
|
||||
return result
|
||||
@@ -225,6 +239,21 @@ class LoginRestServlet(RestServlet):
|
||||
logger.warning(
|
||||
"unknown 3pid identifier medium %s, address %r", medium, address
|
||||
)
|
||||
# We mark that we've failed to log in here, as
|
||||
# `check_password_provider_3pid` might have returned `None` due
|
||||
# to an incorrect password, rather than the account not
|
||||
# existing.
|
||||
#
|
||||
# If it returned None but the 3PID was bound then we won't hit
|
||||
# this code path, which is fine as then the per-user ratelimit
|
||||
# will kick in below.
|
||||
self._failed_attempts_ratelimiter.can_do_action(
|
||||
(medium, address),
|
||||
time_now_s=self._clock.time(),
|
||||
rate_hz=self.hs.config.rc_login_failed_attempts.per_second,
|
||||
burst_count=self.hs.config.rc_login_failed_attempts.burst_count,
|
||||
update=True,
|
||||
)
|
||||
raise LoginError(403, "", errcode=Codes.FORBIDDEN)
|
||||
|
||||
identifier = {"type": "m.id.user", "user": user_id}
|
||||
@@ -236,29 +265,84 @@ class LoginRestServlet(RestServlet):
|
||||
if "user" not in identifier:
|
||||
raise SynapseError(400, "User identifier is missing 'user' key")
|
||||
|
||||
canonical_user_id, callback = yield self.auth_handler.validate_login(
|
||||
identifier["user"], login_submission
|
||||
if identifier["user"].startswith("@"):
|
||||
qualified_user_id = identifier["user"]
|
||||
else:
|
||||
qualified_user_id = UserID(identifier["user"], self.hs.hostname).to_string()
|
||||
|
||||
# Check if we've hit the failed ratelimit (but don't update it)
|
||||
self._failed_attempts_ratelimiter.ratelimit(
|
||||
qualified_user_id.lower(),
|
||||
time_now_s=self._clock.time(),
|
||||
rate_hz=self.hs.config.rc_login_failed_attempts.per_second,
|
||||
burst_count=self.hs.config.rc_login_failed_attempts.burst_count,
|
||||
update=False,
|
||||
)
|
||||
|
||||
result = yield self._register_device_with_callback(
|
||||
try:
|
||||
canonical_user_id, callback = yield self.auth_handler.validate_login(
|
||||
identifier["user"], login_submission
|
||||
)
|
||||
except LoginError:
|
||||
# The user has failed to log in, so we need to update the rate
|
||||
# limiter. Using `can_do_action` avoids us raising a ratelimit
|
||||
# exception and masking the LoginError. The actual ratelimiting
|
||||
# should have happened above.
|
||||
self._failed_attempts_ratelimiter.can_do_action(
|
||||
qualified_user_id.lower(),
|
||||
time_now_s=self._clock.time(),
|
||||
rate_hz=self.hs.config.rc_login_failed_attempts.per_second,
|
||||
burst_count=self.hs.config.rc_login_failed_attempts.burst_count,
|
||||
update=True,
|
||||
)
|
||||
raise
|
||||
|
||||
result = yield self._complete_login(
|
||||
canonical_user_id, login_submission, callback
|
||||
)
|
||||
return result
|
||||
|
||||
@defer.inlineCallbacks
|
||||
def _register_device_with_callback(self, user_id, login_submission, callback=None):
|
||||
""" Registers a device with a given user_id. Optionally run a callback
|
||||
function after registration has completed.
|
||||
def _complete_login(
|
||||
self, user_id, login_submission, callback=None, create_non_existant_users=False
|
||||
):
|
||||
"""Called when we've successfully authed the user and now need to
|
||||
actually login them in (e.g. create devices). This gets called on
|
||||
all succesful logins.
|
||||
|
||||
Applies the ratelimiting for succesful login attempts against an
|
||||
account.
|
||||
|
||||
Args:
|
||||
user_id (str): ID of the user to register.
|
||||
login_submission (dict): Dictionary of login information.
|
||||
callback (func|None): Callback function to run after registration.
|
||||
create_non_existant_users (bool): Whether to create the user if
|
||||
they don't exist. Defaults to False.
|
||||
|
||||
Returns:
|
||||
result (Dict[str,str]): Dictionary of account information after
|
||||
successful registration.
|
||||
"""
|
||||
|
||||
# Before we actually log them in we check if they've already logged in
|
||||
# too often. This happens here rather than before as we don't
|
||||
# necessarily know the user before now.
|
||||
self._account_ratelimiter.ratelimit(
|
||||
user_id.lower(),
|
||||
time_now_s=self._clock.time(),
|
||||
rate_hz=self.hs.config.rc_login_account.per_second,
|
||||
burst_count=self.hs.config.rc_login_account.burst_count,
|
||||
update=True,
|
||||
)
|
||||
|
||||
if create_non_existant_users:
|
||||
user_id = yield self.auth_handler.check_user_exists(user_id)
|
||||
if not user_id:
|
||||
user_id = yield self.registration_handler.register_user(
|
||||
localpart=UserID.from_string(user_id).localpart
|
||||
)
|
||||
|
||||
device_id = login_submission.get("device_id")
|
||||
initial_display_name = login_submission.get("initial_device_display_name")
|
||||
device_id, access_token = yield self.registration_handler.register_device(
|
||||
@@ -285,7 +369,7 @@ class LoginRestServlet(RestServlet):
|
||||
token
|
||||
)
|
||||
|
||||
result = yield self._register_device_with_callback(user_id, login_submission)
|
||||
result = yield self._complete_login(user_id, login_submission)
|
||||
return result
|
||||
|
||||
@defer.inlineCallbacks
|
||||
@@ -313,15 +397,8 @@ class LoginRestServlet(RestServlet):
|
||||
raise LoginError(401, "Invalid JWT", errcode=Codes.UNAUTHORIZED)
|
||||
|
||||
user_id = UserID(user, self.hs.hostname).to_string()
|
||||
|
||||
registered_user_id = yield self.auth_handler.check_user_exists(user_id)
|
||||
if not registered_user_id:
|
||||
registered_user_id = yield self.registration_handler.register_user(
|
||||
localpart=user
|
||||
)
|
||||
|
||||
result = yield self._register_device_with_callback(
|
||||
registered_user_id, login_submission
|
||||
result = yield self._complete_login(
|
||||
user_id, login_submission, create_non_existant_users=True
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
@@ -122,7 +122,7 @@ class PreviewUrlResource(DirectServeResource):
|
||||
pattern = entry[attrib]
|
||||
value = getattr(url_tuple, attrib)
|
||||
logger.debug(
|
||||
"Matching attrib '%s' with value '%s' against" " pattern '%s'",
|
||||
"Matching attrib '%s' with value '%s' against pattern '%s'",
|
||||
attrib,
|
||||
value,
|
||||
pattern,
|
||||
|
||||
@@ -54,7 +54,7 @@ class ConsentServerNotices(object):
|
||||
)
|
||||
if "body" not in self._server_notice_content:
|
||||
raise ConfigError(
|
||||
"user_consent server_notice_consent must contain a 'body' " "key."
|
||||
"user_consent server_notice_consent must contain a 'body' key."
|
||||
)
|
||||
|
||||
self._consent_uri_builder = ConsentURIBuilder(hs.config)
|
||||
|
||||
@@ -851,7 +851,7 @@ class SQLBaseStore(object):
|
||||
allvalues.update(values)
|
||||
latter = "UPDATE SET " + ", ".join(k + "=EXCLUDED." + k for k in values)
|
||||
|
||||
sql = ("INSERT INTO %s (%s) VALUES (%s) " "ON CONFLICT (%s) DO %s") % (
|
||||
sql = ("INSERT INTO %s (%s) VALUES (%s) ON CONFLICT (%s) DO %s") % (
|
||||
table,
|
||||
", ".join(k for k in allvalues),
|
||||
", ".join("?" for _ in allvalues),
|
||||
|
||||
@@ -380,7 +380,7 @@ class DeviceInboxStore(DeviceInboxWorkerStore, DeviceInboxBackgroundUpdateStore)
|
||||
devices = list(messages_by_device.keys())
|
||||
if len(devices) == 1 and devices[0] == "*":
|
||||
# Handle wildcard device_ids.
|
||||
sql = "SELECT device_id FROM devices" " WHERE user_id = ?"
|
||||
sql = "SELECT device_id FROM devices WHERE user_id = ?"
|
||||
txn.execute(sql, (user_id,))
|
||||
message_json = json.dumps(messages_by_device["*"])
|
||||
for row in txn:
|
||||
|
||||
@@ -138,9 +138,9 @@ class EndToEndKeyWorkerStore(SQLBaseStore):
|
||||
result.setdefault(user_id, {})[device_id] = None
|
||||
|
||||
# get signatures on the device
|
||||
signature_sql = (
|
||||
"SELECT * " " FROM e2e_cross_signing_signatures " " WHERE %s"
|
||||
) % (" OR ".join("(" + q + ")" for q in signature_query_clauses))
|
||||
signature_sql = ("SELECT * FROM e2e_cross_signing_signatures WHERE %s") % (
|
||||
" OR ".join("(" + q + ")" for q in signature_query_clauses)
|
||||
)
|
||||
|
||||
txn.execute(signature_sql, signature_query_params)
|
||||
rows = self.cursor_to_dict(txn)
|
||||
|
||||
@@ -713,9 +713,7 @@ class EventsStore(
|
||||
|
||||
metadata_json = encode_json(event.internal_metadata.get_dict())
|
||||
|
||||
sql = (
|
||||
"UPDATE event_json SET internal_metadata = ?" " WHERE event_id = ?"
|
||||
)
|
||||
sql = "UPDATE event_json SET internal_metadata = ? WHERE event_id = ?"
|
||||
txn.execute(sql, (metadata_json, event.event_id))
|
||||
|
||||
# Add an entry to the ex_outlier_stream table to replicate the
|
||||
@@ -732,7 +730,7 @@ class EventsStore(
|
||||
},
|
||||
)
|
||||
|
||||
sql = "UPDATE events SET outlier = ?" " WHERE event_id = ?"
|
||||
sql = "UPDATE events SET outlier = ? WHERE event_id = ?"
|
||||
txn.execute(sql, (False, event.event_id))
|
||||
|
||||
# Update the event_backward_extremities table now that this
|
||||
@@ -1479,7 +1477,7 @@ class EventsStore(
|
||||
|
||||
# We do joins against events_to_purge for e.g. calculating state
|
||||
# groups to purge, etc., so lets make an index.
|
||||
txn.execute("CREATE INDEX events_to_purge_id" " ON events_to_purge(event_id)")
|
||||
txn.execute("CREATE INDEX events_to_purge_id ON events_to_purge(event_id)")
|
||||
|
||||
txn.execute("SELECT event_id, should_delete FROM events_to_purge")
|
||||
event_rows = txn.fetchall()
|
||||
|
||||
@@ -55,7 +55,7 @@ class FilteringStore(SQLBaseStore):
|
||||
if filter_id_response is not None:
|
||||
return filter_id_response[0]
|
||||
|
||||
sql = "SELECT MAX(filter_id) FROM user_filters " "WHERE user_id = ?"
|
||||
sql = "SELECT MAX(filter_id) FROM user_filters WHERE user_id = ?"
|
||||
txn.execute(sql, (user_localpart,))
|
||||
max_id = txn.fetchone()[0]
|
||||
if max_id is None:
|
||||
|
||||
@@ -337,7 +337,7 @@ class MediaRepositoryStore(MediaRepositoryBackgroundUpdateStore):
|
||||
if len(media_ids) == 0:
|
||||
return
|
||||
|
||||
sql = "DELETE FROM local_media_repository_url_cache" " WHERE media_id = ?"
|
||||
sql = "DELETE FROM local_media_repository_url_cache WHERE media_id = ?"
|
||||
|
||||
def _delete_url_cache_txn(txn):
|
||||
txn.executemany(sql, [(media_id,) for media_id in media_ids])
|
||||
@@ -365,11 +365,11 @@ class MediaRepositoryStore(MediaRepositoryBackgroundUpdateStore):
|
||||
return
|
||||
|
||||
def _delete_url_cache_media_txn(txn):
|
||||
sql = "DELETE FROM local_media_repository" " WHERE media_id = ?"
|
||||
sql = "DELETE FROM local_media_repository WHERE media_id = ?"
|
||||
|
||||
txn.executemany(sql, [(media_id,) for media_id in media_ids])
|
||||
|
||||
sql = "DELETE FROM local_media_repository_thumbnails" " WHERE media_id = ?"
|
||||
sql = "DELETE FROM local_media_repository_thumbnails WHERE media_id = ?"
|
||||
|
||||
txn.executemany(sql, [(media_id,) for media_id in media_ids])
|
||||
|
||||
|
||||
@@ -377,9 +377,7 @@ class RegistrationWorkerStore(SQLBaseStore):
|
||||
"""
|
||||
|
||||
def f(txn):
|
||||
sql = (
|
||||
"SELECT name, password_hash FROM users" " WHERE lower(name) = lower(?)"
|
||||
)
|
||||
sql = "SELECT name, password_hash FROM users WHERE lower(name) = lower(?)"
|
||||
txn.execute(sql, (user_id,))
|
||||
return dict(txn)
|
||||
|
||||
|
||||
@@ -616,7 +616,7 @@ class StreamWorkerStore(EventsWorkerStore, SQLBaseStore):
|
||||
|
||||
def _get_max_topological_txn(self, txn, room_id):
|
||||
txn.execute(
|
||||
"SELECT MAX(topological_ordering) FROM events" " WHERE room_id = ?",
|
||||
"SELECT MAX(topological_ordering) FROM events WHERE room_id = ?",
|
||||
(room_id,),
|
||||
)
|
||||
|
||||
|
||||
@@ -83,9 +83,7 @@ class TagsWorkerStore(AccountDataWorkerStore):
|
||||
)
|
||||
|
||||
def get_tag_content(txn, tag_ids):
|
||||
sql = (
|
||||
"SELECT tag, content" " FROM room_tags" " WHERE user_id=? AND room_id=?"
|
||||
)
|
||||
sql = "SELECT tag, content FROM room_tags WHERE user_id=? AND room_id=?"
|
||||
results = []
|
||||
for stream_id, user_id, room_id in tag_ids:
|
||||
txn.execute(sql, (user_id, room_id))
|
||||
|
||||
@@ -447,7 +447,7 @@ def _apply_module_schema_files(cur, database_engine, modname, names_and_streams)
|
||||
# Mark as done.
|
||||
cur.execute(
|
||||
database_engine.convert_param_style(
|
||||
"INSERT INTO applied_module_schemas (module_name, file)" " VALUES (?,?)"
|
||||
"INSERT INTO applied_module_schemas (module_name, file) VALUES (?,?)"
|
||||
),
|
||||
(modname, name),
|
||||
)
|
||||
|
||||
@@ -88,9 +88,12 @@ class PaginationConfig(object):
|
||||
raise SynapseError(400, "Invalid request.")
|
||||
|
||||
def __repr__(self):
|
||||
return (
|
||||
"PaginationConfig(from_tok=%r, to_tok=%r," " direction=%r, limit=%r)"
|
||||
) % (self.from_token, self.to_token, self.direction, self.limit)
|
||||
return ("PaginationConfig(from_tok=%r, to_tok=%r, direction=%r, limit=%r)") % (
|
||||
self.from_token,
|
||||
self.to_token,
|
||||
self.direction,
|
||||
self.limit,
|
||||
)
|
||||
|
||||
def get_source_config(self, source_name):
|
||||
keyname = "%s_key" % source_name
|
||||
|
||||
@@ -16,13 +16,16 @@
|
||||
import functools
|
||||
import inspect
|
||||
import logging
|
||||
import math
|
||||
import threading
|
||||
from collections import deque
|
||||
from typing import Any, Tuple, Union, cast
|
||||
from weakref import WeakValueDictionary
|
||||
|
||||
from six import itervalues
|
||||
|
||||
from prometheus_client import Gauge
|
||||
from bloom_filter import BloomFilter
|
||||
from prometheus_client import Gauge, Histogram
|
||||
from typing_extensions import Protocol
|
||||
|
||||
from twisted.internet import defer
|
||||
@@ -38,6 +41,13 @@ from . import register_cache
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
cache_size_counts = Histogram(
|
||||
"synapse_util_caches_miss_rate",
|
||||
"",
|
||||
["name"],
|
||||
buckets=[0.25, 0.5, 0.75, 1.0, 1.25, 1.5, 1.75, 2, "+Inf"],
|
||||
)
|
||||
|
||||
CacheKey = Union[Tuple, Any]
|
||||
|
||||
|
||||
@@ -87,6 +97,8 @@ class Cache(object):
|
||||
"thread",
|
||||
"metrics",
|
||||
"_pending_deferred_cache",
|
||||
"ratio_tracking",
|
||||
"ratio_metric",
|
||||
)
|
||||
|
||||
def __init__(self, name, max_entries=1000, keylen=1, tree=False, iterable=False):
|
||||
@@ -104,6 +116,9 @@ class Cache(object):
|
||||
self.name = name
|
||||
self.keylen = keylen
|
||||
self.thread = None
|
||||
|
||||
self.ratio_tracking = CacheHitRatioTracking(max_entries)
|
||||
|
||||
self.metrics = register_cache(
|
||||
"cache",
|
||||
name,
|
||||
@@ -111,6 +126,8 @@ class Cache(object):
|
||||
collect_callback=self._metrics_collection_callback,
|
||||
)
|
||||
|
||||
self.ratio_metric = cache_size_counts.labels(name)
|
||||
|
||||
def _on_evicted(self, evicted_count):
|
||||
self.metrics.inc_evictions(evicted_count)
|
||||
|
||||
@@ -148,6 +165,11 @@ class Cache(object):
|
||||
self.metrics.inc_hits()
|
||||
return val.deferred
|
||||
|
||||
ratio = self.ratio_tracking.add(str(hash(key)))
|
||||
if ratio is None:
|
||||
ratio = 10
|
||||
self.ratio_metric.observe(ratio)
|
||||
|
||||
val = self.cache.get(key, _CacheSentinel, callbacks=callbacks)
|
||||
if val is not _CacheSentinel:
|
||||
self.metrics.inc_hits()
|
||||
@@ -222,6 +244,9 @@ class Cache(object):
|
||||
|
||||
def prefill(self, key, value, callback=None):
|
||||
callbacks = [callback] if callback else []
|
||||
|
||||
self.ratio_tracking.add(str(hash(key)))
|
||||
|
||||
self.cache.set(key, value, callbacks=callbacks)
|
||||
|
||||
def invalidate(self, key):
|
||||
@@ -724,3 +749,76 @@ def cachedList(cached_method_name, list_name, num_args=None, inlineCallbacks=Fal
|
||||
num_args=num_args,
|
||||
inlineCallbacks=inlineCallbacks,
|
||||
)
|
||||
|
||||
|
||||
class CacheHitRatioTracking(object):
|
||||
def __init__(self, target_size, buckets=20, error_rate=0.01):
|
||||
self._bucket_size = 2 * target_size / buckets
|
||||
self._error_rate = error_rate
|
||||
|
||||
self._target_size = target_size
|
||||
|
||||
self._buckets = deque(
|
||||
(
|
||||
[
|
||||
BloomFilter(
|
||||
max_elements=self._target_size, error_rate=self._error_rate
|
||||
),
|
||||
0,
|
||||
]
|
||||
for _ in range(buckets)
|
||||
),
|
||||
maxlen=buckets,
|
||||
)
|
||||
|
||||
def add(self, key):
|
||||
found_in_bucket = None
|
||||
|
||||
for i, (bucket, _) in enumerate(self._buckets):
|
||||
if key in bucket:
|
||||
found_in_bucket = i
|
||||
break
|
||||
else:
|
||||
self._buckets[i][1] += 1
|
||||
|
||||
self._buckets[0][0].add(key)
|
||||
|
||||
while self._buckets[-1][1] > 2 * self._target_size:
|
||||
self._buckets.pop()
|
||||
|
||||
if self._buckets[0][1] > self._bucket_size:
|
||||
self._buckets.appendleft(
|
||||
[
|
||||
BloomFilter(
|
||||
max_elements=self._target_size, error_rate=self._error_rate
|
||||
),
|
||||
0,
|
||||
]
|
||||
)
|
||||
|
||||
if found_in_bucket is not None:
|
||||
return self._buckets[found_in_bucket][1] / self._target_size
|
||||
|
||||
|
||||
def _count_set_bits(n):
|
||||
count = 0
|
||||
while n:
|
||||
count += n & 1
|
||||
n >>= 1
|
||||
return count
|
||||
|
||||
|
||||
def _num_bits_set(bloom):
|
||||
count = 0
|
||||
for c in bloom.backend.array_:
|
||||
if c == 0:
|
||||
continue
|
||||
count += _count_set_bits(c)
|
||||
return count
|
||||
|
||||
|
||||
def _get_estimate_bloom_size(bloom):
|
||||
X = _num_bits_set(bloom)
|
||||
m = bloom.num_bits_m
|
||||
k = bloom.num_probes_k
|
||||
return -(m / k) * math.log(1 - X / m)
|
||||
|
||||
Reference in New Issue
Block a user