Hi
This post is about the work summary pertaining to my GSoC 2016 project "Message Queues based Archiver".
The project involved creating archiving interfaces for the GNU Mailman which are based on message queues.
The initial stage involved investigating how archiving happens in Mailman, and how it needs to be tweaked with minimum disturbance in order to accommodate message queue based interfaces. The challenging part was to fit in one interface multiple archives pattern in an one interface one archive pattern. This involved changing the naming policy of the interface instances in Mailman core.
After investigating upon various api's offerred for message queuing backends, RabbitMQ was found to be the ideal choice for the use case. Though initially a simple message queue based interface(MQArchiver) was developed, it was later extended to incorporate a publisher-subscriber model based interface named PubMQArchiver, which feeds mails to a common channel, and from where mailms may be picked up by a variety of Web Applications.
After incorporating different message patterns(such as direct queueing and publisher subscriber), the interface was extended to incorporate pluggable backends, where one can further develop a Backend class calling api calls in other message queueing api's based off a Base backend class. This makes our design open to accomodate other api's than RabbitMQ as well.
The archiver interfaces are configurable to set message queueing server host, exchange_names for common channels, backends and backend specific parameters. In case of failures(of missing configuration or parameters), the archiver falls back on default configuration(which is based on RabbitMQ backend with server host at 'localhost' and exchange name 'pubsub'.
A feature to delete archiver entries in database unable to access their archiver instances (called "Stale Archivers") was added to postorious(GUI).
Below are the links to repositories with my work :
1. I worked on developing archiving interfaces for GNU Mailman. These are located at :
https://gitlab.com/anirudhdahiya9/MailmanMQArchiver
2. My work involved bringing in changes to the GUI for deletion of "Stale Archivers", i.e. archivers whose database entries cannot be mapped to active archiver instances(called "System Archivers").
https://gitlab.com/anirudhdahiya9/postorius/tree/stale-archivers
3. This needed slight changes to mailmanclient as well, in order for requests to be mapped between postorius and the REST.
https://gitlab.com/anirudhdahiya9/mailmanclient/tree/stale-archivers
4. My project required slightly changing the core mailman archiving procedure.
https://gitlab.com/anirudhdahiya9/mailman/tree/MQArchiver
I must mention, that all this was possible with the constant guidance of my project mentor Florian Fuchs, and Barry Warsaw, among other contributors at GNU Mailman.
Thanks
Anirudh Dahiya
This post is about the work summary pertaining to my GSoC 2016 project "Message Queues based Archiver".
The project involved creating archiving interfaces for the GNU Mailman which are based on message queues.
The initial stage involved investigating how archiving happens in Mailman, and how it needs to be tweaked with minimum disturbance in order to accommodate message queue based interfaces. The challenging part was to fit in one interface multiple archives pattern in an one interface one archive pattern. This involved changing the naming policy of the interface instances in Mailman core.
After investigating upon various api's offerred for message queuing backends, RabbitMQ was found to be the ideal choice for the use case. Though initially a simple message queue based interface(MQArchiver) was developed, it was later extended to incorporate a publisher-subscriber model based interface named PubMQArchiver, which feeds mails to a common channel, and from where mailms may be picked up by a variety of Web Applications.
After incorporating different message patterns(such as direct queueing and publisher subscriber), the interface was extended to incorporate pluggable backends, where one can further develop a Backend class calling api calls in other message queueing api's based off a Base backend class. This makes our design open to accomodate other api's than RabbitMQ as well.
The archiver interfaces are configurable to set message queueing server host, exchange_names for common channels, backends and backend specific parameters. In case of failures(of missing configuration or parameters), the archiver falls back on default configuration(which is based on RabbitMQ backend with server host at 'localhost' and exchange name 'pubsub'.
A feature to delete archiver entries in database unable to access their archiver instances (called "Stale Archivers") was added to postorious(GUI).
Below are the links to repositories with my work :
1. I worked on developing archiving interfaces for GNU Mailman. These are located at :
https://gitlab.com/anirudhdahiya9/MailmanMQArchiver
2. My work involved bringing in changes to the GUI for deletion of "Stale Archivers", i.e. archivers whose database entries cannot be mapped to active archiver instances(called "System Archivers").
https://gitlab.com/anirudhdahiya9/postorius/tree/stale-archivers
3. This needed slight changes to mailmanclient as well, in order for requests to be mapped between postorius and the REST.
https://gitlab.com/anirudhdahiya9/mailmanclient/tree/stale-archivers
4. My project required slightly changing the core mailman archiving procedure.
https://gitlab.com/anirudhdahiya9/mailman/tree/MQArchiver
I must mention, that all this was possible with the constant guidance of my project mentor Florian Fuchs, and Barry Warsaw, among other contributors at GNU Mailman.
Thanks
Anirudh Dahiya
No comments:
Post a Comment