kaslo/synapse
1
0
Fork 0
Code Wiki Activity

Merge branch 'develop' into dmr/pyproject-poetry

Browse Source 
Signed-off-by: Dan Callahan <danc@element.io>
This commit is contained in:
Dan Callahan
2022-04-07 00:37:27 +01:00
parent f0d72acebf 350062661c
commit 60a66716f8
200 changed files with 6095 additions and 5282 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
docs/SUMMARY.md
View File

@@ -45,6 +45,7 @@
- [Account validity callbacks](modules/account_validity_callbacks.md)
- [Password auth provider callbacks](modules/password_auth_provider_callbacks.md)
- [Background update controller callbacks](modules/background_update_controller_callbacks.md)
- [Account data callbacks](modules/account_data_callbacks.md)
- [Porting a legacy module to the new interface](modules/porting_legacy_module.md)
- [Workers](workers.md)
- [Using `synctl` with Workers](synctl_workers.md)

14
docs/development/contributing_guide.md
View File

@@ -201,9 +201,10 @@ To do so, [configure Postgres](../postgres.md) and run `trial` with the
following environment variables matching your configuration:
- `SYNAPSE_POSTGRES` to anything nonempty
- `SYNAPSE_POSTGRES_HOST`
- `SYNAPSE_POSTGRES_USER`
- `SYNAPSE_POSTGRES_PASSWORD`
- `SYNAPSE_POSTGRES_HOST` (optional if it's the default: UNIX socket)
- `SYNAPSE_POSTGRES_PORT` (optional if it's the default: 5432)
- `SYNAPSE_POSTGRES_USER` (optional if using a UNIX socket)
- `SYNAPSE_POSTGRES_PASSWORD` (optional if using a UNIX socket)
For example:
@@ -215,6 +216,13 @@ export SYNAPSE_POSTGRES_PASSWORD=mydevenvpassword
trial
```
You don't need to specify the host, user, port or password if your Postgres
server is set to authenticate you over the UNIX socket (i.e. if the `psql` command
works without further arguments).
Your Postgres account needs to be able to create databases.
## Run the integration tests ([Sytest](https://github.com/matrix-org/sytest)).
The integration tests are a more comprehensive suite of tests. They

3
docs/development/room-dag-concepts.md
View File

@@ -39,7 +39,8 @@ yet correlated to the DAG.
Outliers typically arise when we fetch the auth chain or state for a given
event. When that happens, we just grab the events in the state/auth chain,
without calculating the state at those events, or backfilling their
`prev_events`.
`prev_events`. Since we don't have the state at any events fetched in that
way, we mark them as outliers.
So, typically, we won't have the `prev_events` of an `outlier` in the database,
(though it's entirely possible that we *might* have them for some other

106
docs/modules/account_data_callbacks.md Normal file
View File

@@ -0,0 +1,106 @@
# Account data callbacks
Account data callbacks allow module developers to react to changes of the account data
of local users. Account data callbacks can be registered using the module API's
`register_account_data_callbacks` method.
## Callbacks
The available account data callbacks are:
### `on_account_data_updated`
_First introduced in Synapse v1.57.0_
```python
async def on_account_data_updated(
user_id: str,
room_id: Optional[str],
account_data_type: str,
content: "synapse.module_api.JsonDict",
) -> None:
```
Called after user's account data has been updated. The module is given the
Matrix ID of the user whose account data is changing, the room ID the data is associated
with, the type associated with the change, as well as the new content. If the account
data is not associated with a specific room, then the room ID is `None`.
This callback is triggered when new account data is added or when the data associated with
a given type (and optionally room) changes. This includes deletion, since in Matrix,
deleting account data consists of replacing the data associated with a given type
(and optionally room) with an empty dictionary (`{}`).
Note that this doesn't trigger when changing the tags associated with a room, as these are
processed separately by Synapse.
If multiple modules implement this callback, Synapse runs them all in order.
## Example
The example below is a module that implements the `on_account_data_updated` callback, and
sends an event to an audit room when a user changes their account data.
```python
import json
import attr
from typing import Any, Dict, Optional
from synapse.module_api import JsonDict, ModuleApi
from synapse.module_api.errors import ConfigError
@attr.s(auto_attribs=True)
class CustomAccountDataConfig:
audit_room: str
sender: str
class CustomAccountDataModule:
def __init__(self, config: CustomAccountDataConfig, api: ModuleApi):
self.api = api
self.config = config
self.api.register_account_data_callbacks(
on_account_data_updated=self.log_new_account_data,
)
@staticmethod
def parse_config(config: Dict[str, Any]) -> CustomAccountDataConfig:
def check_in_config(param: str):
if param not in config:
raise ConfigError(f"'{param}' is required")
check_in_config("audit_room")
check_in_config("sender")
return CustomAccountDataConfig(
audit_room=config["audit_room"],
sender=config["sender"],
)
async def log_new_account_data(
self,
user_id: str,
room_id: Optional[str],
account_data_type: str,
content: JsonDict,
) -> None:
content_raw = json.dumps(content)
msg_content = f"{user_id} has changed their account data for type {account_data_type} to: {content_raw}"
if room_id is not None:
msg_content += f" (in room {room_id})"
await self.api.create_and_send_event_into_room(
{
"room_id": self.config.audit_room,
"sender": self.config.sender,
"type": "m.room.message",
"content": {
"msgtype": "m.text",
"body": msg_content
}
}
)
```

2
docs/modules/writing_a_module.md
View File

@@ -33,7 +33,7 @@ A module can implement the following static method:
```python
@staticmethod
def parse_config(config: dict) -> dict
def parse_config(config: dict) -> Any
```
This method is given a dictionary resulting from parsing the YAML configuration for the

79
docs/structured_logging.md
View File

@@ -78,82 +78,3 @@ loggers:
The above logging config will set Synapse as 'INFO' logging level by default,
with the SQL layer at 'WARNING', and will log JSON formatted messages to a
remote endpoint at 10.1.2.3:9999.
## Upgrading from legacy structured logging configuration
Versions of Synapse prior to v1.54.0 automatically converted the legacy
structured logging configuration, which was deprecated in v1.23.0, to the standard
library logging configuration.
The following reference can be used to update your configuration. Based on the
drain `type`, we can pick a new handler:
1. For a type of `console`, `console_json`, or `console_json_terse`: a handler
with a class of `logging.StreamHandler` and a `stream` of `ext://sys.stdout`
or `ext://sys.stderr` should be used.
2. For a type of `file` or `file_json`: a handler of `logging.FileHandler` with
a location of the file path should be used.
3. For a type of `network_json_terse`: a handler of `synapse.logging.RemoteHandler`
with the host and port should be used.
Then based on the drain `type` we can pick a new formatter:
1. For a type of `console` or `file` no formatter is necessary.
2. For a type of `console_json` or `file_json`: a formatter of
`synapse.logging.JsonFormatter` should be used.
3. For a type of `console_json_terse` or `network_json_terse`: a formatter of
`synapse.logging.TerseJsonFormatter` should be used.
For each new handler and formatter they should be added to the logging configuration
and then assigned to either a logger or the root logger.
An example legacy configuration:
```yaml
structured: true
loggers:
synapse:
level: INFO
synapse.storage.SQL:
level: WARNING
drains:
console:
type: console
location: stdout
file:
type: file_json
location: homeserver.log
```
Would be converted into a new configuration:
```yaml
version: 1
formatters:
json:
class: synapse.logging.JsonFormatter
handlers:
console:
class: logging.StreamHandler
stream: ext://sys.stdout
file:
class: logging.FileHandler
formatter: json
filename: homeserver.log
loggers:
synapse:
level: INFO
handlers: [console, file]
synapse.storage.SQL:
level: WARNING
```
The new logging configuration is a bit more verbose, but significantly more
flexible. It allows for configuration that were not previously possible, such as
sending plain logs over the network, or using different handlers for different
modules.

28
docs/upgrade.md
View File

@@ -88,8 +88,26 @@ process, for example:
dpkg -i matrix-synapse-py3_1.3.0+stretch1_amd64.deb
```
# Upgrading to v1.57.0
## Changes to database schema for application services
Synapse v1.57.0 includes a [change](https://github.com/matrix-org/synapse/pull/12209) to the
way transaction IDs are managed for application services. If your deployment uses a dedicated
worker for application service traffic, **it must be stopped** when the database is upgraded
(which normally happens when the main process is upgraded), to ensure the change is made safely
without any risk of reusing transaction IDs.
Deployments which do not use separate worker processes can be upgraded as normal. Similarly,
deployments where no applciation services are in use can be upgraded as normal.
# Upgrading to v1.56.0
## Open registration without verification is now disabled by default
Synapse will refuse to start if registration is enabled without email, captcha, or token-based verification unless the new config
flag `enable_registration_without_verification` is set to "true".
## Groups/communities feature has been deprecated
The non-standard groups/communities feature in Synapse has been deprecated and will
@@ -111,12 +129,6 @@ for more information and instructions on how to fix a database with incorrect va
# Upgrading to v1.55.0
## Open registration without verification is now disabled by default
Synapse will refuse to start if registration is enabled without email, captcha, or token-based verification unless the new config
flag `enable_registration_without_verification` is set to "true".
## `synctl` script has been moved
The `synctl` script
@@ -151,7 +163,7 @@ please upgrade Mjolnir to version 1.3.2 or later before upgrading Synapse.
This release removes support for the `structured: true` logging configuration
which was deprecated in Synapse v1.23.0. If your logging configuration contains
`structured: true` then it should be modified based on the
[structured logging documentation](structured_logging.md).
[structured logging documentation](https://matrix-org.github.io/synapse/v1.56/structured_logging.html#upgrading-from-legacy-structured-logging-configuration).
# Upgrading to v1.53.0
@@ -768,7 +780,7 @@ lock down external access to the Admin API endpoints.
This release deprecates use of the `structured: true` logging
configuration for structured logging. If your logging configuration
contains `structured: true` then it should be modified based on the
[structured logging documentation](structured_logging.md).
[structured logging documentation](https://matrix-org.github.io/synapse/v1.56/structured_logging.html#upgrading-from-legacy-structured-logging-configuration).
The `structured` and `drains` logging options are now deprecated and
should be replaced by standard logging configuration of `handlers` and

2
docs/workers.md
View File

@@ -27,7 +27,7 @@ feeds streams of newly written data between processes so they can be kept in
sync with the database state.
When configured to do so, Synapse uses a
[Redis pub/sub channel](https://redis.io/topics/pubsub) to send the replication
[Redis pub/sub channel](https://redis.io/docs/manual/pubsub/) to send the replication
stream between all configured Synapse processes. Additionally, processes may
make HTTP requests to each other, primarily for operations which need to wait
for a reply ─ such as sending an event.