merge prod

Reviewed-on: https://git.ugnet.gay/CrossTalk/azul/pulls/1

README.md

@@ -1,6 +1,6 @@
-# Azul
+# AZUL
 
-Azul is the main server software for the CrossTalk service; a chat service planning to support as many messaging protocols as possible, both retro and modern. This service acts as the backbone for all chat-related services that are a part of the undergr0und.
+AZUL is the main server software for the CrossTalk service; a chat service planning to support as many messaging protocols as possible, both retro and modern. Based on the legacy Escargot codebase.
 
 ## Protocol support
 
@@ -12,27 +12,27 @@ Currently, MSNP2 through MSNP18 are fully implemented, with an MSNP19 - MSNP21 i
 
 - Circles/Groups for Windows Live Messenger 2009 are fully implemented, however, managing members is expected to be done from the CrossTalk website. You can use `script/managecircle.py` to perform a few actions on a Circle and its users (run `python script/managecircle.py -h` to see instructions)
 
-- Windows Live Messenger 2011/2012 supports authentication, viewing your contacts list, presence updates, and MsgrConfig. Most other functionality is either unimplemented or untested
+- Windows Live Messenger 2011/2012 supports authentication, viewing your contacts list, presence updates, and MsgrConfig, and instant messaging. Most other functionality is either unimplemented or untested. Peer-to-peer functionality (display picture syncing, file transfers, activites/games etc) currently does not interop between 2011/2012 and older versions; it only works with other 2011/2012 users.
 
 ### YMSG (used by Yahoo! Messenger)
 
-As of now, only YMSG9 through YMSG15 are implemented, with some caveats:
+As of now, YMSG9 through YMSG15 are implemented, with some caveats:
 
-- If you want to log in to Yahoo! Messenger < 7.0, an MD5crypt-encoded password hash must be stored
+- If you want to log in, an MD5crypt-encoded password hash must be stored
 
 - Conferences are currently not working
 
 - Yahoo! Messenger 7.0 does not log in for currently unknown reasons. 5.0, 5.5, 5.6, 6.0, 7.5, 8.0, and 8.1 were all tested to work, however
 
-- Yahoo! Messneger for Vista is untested, but should work
+- Yahoo! Messneger for Vista also does not work for currently unknown reasons
 
 ### IRC
 
-IRC support is going to be completely reworked with the introduction of chatrooms, but as it stands, it's fairly basic. It offers the ability to create, join, invite people to, leave, and list other people in temporary chats, and to set your nickname, view the MOTD, and set an away message. It also requires `USER`-based login with your account credentials, however, IRC-only guest logins are planned.
+IRC is going to be completely reworked with the introduction of chatrooms, but as it stands, it's fairly basic. It offers the ability to create, join, invite people to, leave, and list other people in temporary chats, and to set your nickname, view the MOTD, and set an away message. It also requires `USER`-based login with your account credentials, however, IRC-only guest logins are planned.
 
 ### OSCAR (used by AOL Instant Messenger and ICQ 2000 - 8)
 
-OSCAR support is basic. Right now, only `FEEDBAG`, `BUDDY`, et cetera are partially implemented, and it supports showing your buddy list as well as presence updates. Messaging currently isn't supported yet. Only the FLAP authentication method and BUCP with the older hashing method (used by AIM 3.5 & 4.x) is implemented. It has been tested to work with AIM 4.3 and 4.8; older versions may be hit or miss. AIM 5.x - 7.x, TOC/TOC2, WebAPI, and ICQ are not supported yet, but support for them is planned and will come in the future.
+OSCAR is basic, but functional. Right now, messaging and buddy list management is implemented. The FLAP and BUCP authentication methods are implemented. AIM 1.0 - 5.9 are supported, but 4.x and 5.x are where most testing is done, so older versions may be hit or miss. AIM 6.x - 7.x, TOC/TOC2, WebAPI, and ICQ are not supported yet, but support for them is planned and will come in the future. Not sure about AIM 8 because of NINA getting exploded.
 
 ## Developers
 

config/bannerimages.json

@@ -6,91 +6,86 @@
 	},
 	{
 		"id": 2,
-		"image": "banner/discord.gif",
-		"link": "https://discord.gg/dumJwXTPxX"
-	},
-	{
-		"id": 3,
 		"image": "banner/amrd24.gif",
 		"link": "https://amrd24.github.io/ctad.html"
 	},
 	{
-		"id": 4,
+		"id": 3,
 		"image": "banner/fortind.gif",
 		"link": "https://www.fort1nd.com"
 	},
 	{
-		"id": 5,
+		"id": 4,
 		"image": "banner/abrefresh.png",
 		"link": "https://www.teamflashcord.com/projects/ab-refresh"
 	},
 	{
-		"id": 6,
+		"id": 5,
 		"image": "banner/cobbclub.png",
 		"link": "http://cobb.club"
 	},
 	{
-		"id": 7,
+		"id": 6,
 		"image": "banner/maymundere.gif",
 		"link": "https://maymundere.org"
 	},
 	{
-		"id": 8,
+		"id": 7,
 		"image": "banner/macsecret.png",
 		"link": "http://macsecret.com"
 	},
 	{
-		"id": 9,
+		"id": 8,
 		"image": "banner/leefy.png",
 		"link": "https://leefymoons-room.net"
 	},
 	{
-		"id": 10,
+		"id": 9,
 		"image": "banner/srcfreak.jpg",
 		"link": "http://srcfreaks.ddns.net"
 	},
 	{
-		"id": 11,
+		"id": 10,
 		"image": "banner/kakworm.gif",
 		"link": "https://fusionstrike.neocities.org"
 	},
 	{
-		"id": 12,
+		"id": 11,
 		"image": "banner/gdsk.gif",
 		"link": "https://gdsk.retrosite.org"
 	},
 	{
-		"id": 13,
+		"id": 12,
 		"image": "banner/kooper.png",
 		"link": "http://kooper.online"
 	},
 	{
-		"id": 14,
+		"id": 13,
 		"image": "banner/imsnp.png",
 		"link": "http://kooper.online/blog/pages/imsnp.html"
 	},
 	{
-		"id": 15,
+		"id": 14,
 		"image": "banner/aomeix.jpg",
 		"link": "http://aomeix.eu.org"
 	},
     {
-		"id": 16,
+		"id": 15,
 		"image": "banner/bluebrixhq.gif",
 		"link": "https://brix.neocities.org"
     },
 	{
-		"id": 17,
+		"id": 16,
 		"image": "banner/thebrothers.png",
 		"link": "https://the-brothers.neocities.org"
     },
 	{
-		"id": 18,
+		"id": 17,
 		"image": "banner/action97.gif",
 		"link": "https://bloxy97.nekoweb.org"
     },
 	{
-		"id": 19,
+		"id": 18,
 		"image": "banner/barco.gif",
 		"link": "https://barcohot.nekoweb.org"
     }

config/textads.json

@@ -7,10 +7,6 @@
 		"caption": "Looking for friends/contacts? Check out our new Member Directory!",
 		"url": "https://crosstalk.im/directory"
 	},
-	{
-		"caption": "Did you know we have a Discord server? Join it now!",
-		"url": "https://discord.gg/dumJwXTPxX"
-	},
 	{
 		"caption": "ClassicConnect - A community of retro tech geeks.",
 		"url": "http://classicconnect.net"

config/whatsnew.json

@@ -3,5 +3,5 @@
 	"url": "https://crosstalk.im",
 	"title": "Update your e-mail if needed",
 	"appName": "CrossTalk News",
-	"content": "Any accounts using a blacklisted email address (such as @crosstalk.im, @escargot.chat, @webtv.zone etc) must change their email address to a real one by July 20, 2026 or your account will be deleted. We are preparing for 0.6’s release and this needs to be done to prevent issues with the upcoming username aliasing/email masking system." 
+	"content": "Any accounts using a blacklisted email address (such as @crosstalk.im, @escargot.chat, @webtv.zone etc) must change their email address to a real one by July 20, 2026 or your account will be deleted. We are preparing for 0.6’s release and this needs to be done to prevent issues with the upcoming aliasing system." 
 }]

core/backend.py

@@ -12,7 +12,7 @@ from .stats import Stats
 from .client import Client
 from .models import (
 	User, UserDetail, Group, ContactList, OIM, Contact, ContactDetail, Circle, CircleRole, CircleState,
-	CircleMembership, Chatroom, UserStatus, TextWithData, MessageData, Substatus, LoginOption, _is_blocking,
+	CircleMembership, Chatroom, UserStatus, TextWithData, MessageData, MessageType, Substatus, LoginOption, _is_blocking,
 )
 from . import error, event
 
@@ -248,8 +248,14 @@ class Backend:
 	def _mark_circle_modified(self, circle: Circle) -> None:
 		self._worklist_sync_circles[circle.chat_id] = circle
 	
+	def util_get_user_email(self, user: User) -> str:
+		return self.user_service.get_user_email(user)
+
+	def util_get_display_email(self, user: User) -> str:
+		return self.user_service.get_display_email(user)
+
 	def util_get_uuid_from_email(self, email: str) -> Optional[str]:
-		return self.user_service.get_uuid(email)
+		return self.user_service.lookup_uuid_by_email(email)
 	
 	def util_get_uuid_from_username(self, username: str) -> Optional[str]:
 		return self.user_service.get_uuid_username(username)
@@ -450,7 +456,7 @@ class Backend:
 	
 	async def _worker_sync_db(self) -> None:
 		while True:
-			await asyncio.sleep(0.0125)
+			await asyncio.sleep(0.04)
 			self._sync_db_impl()
 	
 	def _sync_db_impl(self) -> None:
@@ -468,7 +474,7 @@ class Backend:
 	
 	async def _worker_remove_expired_login_tokens(self) -> None:
 		while True:
-			await asyncio.sleep(0.0125)
+			await asyncio.sleep(0.04)
 			self._remove_expired_login_tokens()
 	
 	def _remove_expired_login_tokens(self) -> None:
@@ -479,7 +485,7 @@ class Backend:
 	
 	async def _worker_sync_circles(self) -> None:
 		while True:
-			await asyncio.sleep(0.0125)
+			await asyncio.sleep(0.04)
 			self._sync_circle_impl()
 	
 	def _sync_circle_impl(self) -> None:
@@ -497,7 +503,7 @@ class Backend:
 	
 	async def _worker_clean_sessions(self) -> None:
 		while True:
-			await asyncio.sleep(0.0125)
+			await asyncio.sleep(0.04)
 			closed = []
 			
 			try:
@@ -521,7 +527,7 @@ class Backend:
 	async def _worker_notify(self) -> None:
 		# Notify relevant `BackendSession`s of status, name, message, media, etc. changes
 		while True:
-			await asyncio.sleep(0.0125)
+			await asyncio.sleep(0.04)
 			try:
 				self._handle_worklist_notify()
 			except:
@@ -566,7 +572,7 @@ class Backend:
 		# Notify relevant `BackendSession`s of status, name, message, media, etc. changes
 		worklist = self._worklist_notify_self
 		while True:
-			await asyncio.sleep(0.0125)
+			await asyncio.sleep(0.04)
 			try:
 				for bs, old_substatus, update_status, update_info in worklist:
 					user = bs.user
@@ -757,7 +763,7 @@ class BackendSession(Session):
 		detail.delete_group(group)
 		for ctc in detail.contacts.values():
 			ctc.remove_from_group(group)
-		print(f'[Contacts] <Info> {user.username} ({user.email}) has deleted group {name} (ID: {group_id})')
+		print(f'[Contacts] <Info> {user.username} ({user.email}) has deleted group {group.name} (ID: {group_id})')
 		self.backend._mark_modified(user)
 	
 	def me_group_edit(self, group_id: str, *, new_name: Optional[str] = None, is_favorite: Optional[bool] = None) -> None:
@@ -1541,7 +1547,7 @@ class ChatSession(Session):
 		stats = self.chat._stats
 		client = self.bs.client
 		
-		if stats:
+		if stats and data.type == MessageType.Chat or MessageType.Nudge or MessageType.Emoticon or MessageType.Ink:
 			stats.on_message_sent(self.user, client)
 			stats.on_user_active(self.user, client)
 		
@@ -1557,7 +1563,7 @@ class ChatSession(Session):
 		stats = self.chat._stats
 		client = self.bs.client
 		
-		if stats:
+		if stats and data.type == MessageType.Chat or MessageType.Nudge or MessageType.Emoticon or MessageType.Ink:
 			stats.on_message_sent(self.user, client)
 			stats.on_user_active(self.user, client)
 		
@@ -1566,7 +1572,7 @@ class ChatSession(Session):
 			if cs_other.user.uuid != user_uuid: continue
 			print(f"[Chat] <Info> {self.user.username} ({self.user.email}) has sent a messasge to {cs_other.user.username} ({cs_other.user.email}) of type {data.type}")
 			cs_other.evt.on_message(data)
-			if stats:
+			if stats and data.type == MessageType.Chat or MessageType.Nudge or MessageType.Emoticon or MessageType.Ink:
 				stats.on_message_received(cs_other.user, client)
 
 def _gen_group_id(detail: UserDetail) -> str:

core/db.py

@@ -46,8 +46,8 @@ class User(WithFrontData):
 	date_created = Col(sa.DateTime, default = datetime.utcnow)
 	date_login = Col(sa.DateTime, nullable = True)
 	uuid = Col(sa.String(50), unique = True)
-	email = Col(sa.String(80))
-	username = Col(sa.String(40))
+	email = Col(sa.String(80, collation='utf8mb4_unicode_ci'), nullable=False, unique=True)
+	username = Col(sa.String(40, collation='utf8mb4_unicode_ci'), nullable=False, unique=True)
 	first_name = Col(sa.String(50), default = 'John')
 	middle_name = Col(sa.String(50), nullable = True)
 	last_name = Col(sa.String(50), default = 'Doe')
@@ -77,6 +77,7 @@ class User(WithFrontData):
 	did_firsttime_email_change = Col(sa.Boolean, default=False)
 	account_verified = Col(sa.Boolean, default=False) # for e-mail address verification
 	lang = Col(sa.String(10), nullable = True)
+	avatar = Col(sa.String(16), nullable = True)
 	profile = relationship("UserProfile", backref="user", uselist=False, cascade="all, delete-orphan")
 	__table_args__ = (sa.Index('email_ci_index', sa.text('(LOWER(email))'), unique = True), sa.Index('username_ci_index', sa.text('(LOWER(username))'), unique = True))
 

core/http.py

@@ -3,7 +3,8 @@ from typing import Any, Dict, Optional
 from aiohttp import web, ClientSession, ClientTimeout, ClientError
 from pathlib import Path
 
-from util.misc import AIOHTTPRunner, _get_avatar_path
+from util.misc import AIOHTTPRunner
+from util.avatar import get_avatar_data
 from core.backend import Backend
 
 from brutus import Brutus
@@ -32,7 +33,8 @@ def register(loop: asyncio.AbstractEventLoop, backend: Backend, *, devmode: bool
 	app.router.add_get('/ads/GetMSNAdImage.asmx', handle_msnbannerimg)
 	app.router.add_get('/svcs/mms/adxml_main.asp', lambda req: handle_bannerad(req, msnxml=True))
 	app.router.add_get('/avatar/{uuid}/static', handle_avatar)
-	app.router.add_get('/avatar/{uuid}/small', lambda req: handle_avatar(req, small = True))
+	app.router.add_get('/avatar/{uuid}/small', lambda req: handle_avatar(req, small=True))
+	app.router.add_get('/avatar/{uuid}/msn', lambda req: handle_avatar(req, msn=True))
 	app.router.add_route('*', '/{path:.*}', handle_notfound)
  
 	return app
@@ -100,8 +102,8 @@ async def handle_bannerad(req: web.Request, yuge: bool = False, sq: bool = False
 		'Content-Type': 'text/xml' if msnxml else 'text/html',
 	}
 		
-	html_response = f'<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"><html style="-ms-overflow-style: none;"><body style="-ms-overflow-style: none;overflow: hidden;padding:0;margin:0;overflow:hidden;-ms-scroll-limit: 0 0 0 0;" scroll=no><a href="{image_link}" style="text-decoration:none;" target="_blank"><img border="0" src="http://static.ugnet.gay/svc/ads/ct/{random_entry["image"]}"></a></body></html>'
-	msn_response = f'<?xml version="1.0"?><ADRSP V="{version}"><image IMG="http://ctsvcs.advertising.ugnet.gay/ads/GetMSNAdImage.asmx?id={id}" ALT="Advertisement" HEIGHT="60" WIDTH="234"/><click CLK="{image_link}" TARGET="_NEW"/></ADRSP>'
+	html_response = f'<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"><html style="-ms-overflow-style: none;"><body style="-ms-overflow-style: none;overflow: hidden;padding:0;margin:0;overflow:hidden;-ms-scroll-limit: 0 0 0 0;" scroll=no><a href="{image_link}" style="text-decoration:none;" target="_blank"><img border="0" src="http://{settings.STATIC_HOST}/svc/ads/ct/{random_entry["image"]}"></a></body></html>'
+	msn_response = f'<?xml version="1.0"?><ADRSP V="{version}"><image IMG="http://{settings.AD_HOST}/ads/GetMSNAdImage.asmx?id={id}" ALT="Advertisement" HEIGHT="60" WIDTH="234"/><click CLK="{image_link}" TARGET="_NEW"/></ADRSP>'
 	return web.HTTPOk(body=msn_response if msnxml else html_response, headers=headers)
 
 async def handle_msnbannerimg(req: web.Request) -> web.Response:
@@ -119,7 +121,7 @@ async def handle_msnbannerimg(req: web.Request) -> web.Response:
 	else:
 		entry = random.choice(data)
 
-	image_url = f"http://static.ugnet.gay/svc/ads/ct/{entry['image']}"
+	image_url = f"http://{settings.STATIC_HOST}/svc/ads/ct/{entry['image']}"
 
 	async with ClientSession() as sess:
 		async with sess.get(image_url) as resp:
@@ -152,46 +154,22 @@ async def handle_blankok(req: web.Request) -> web.Response:
 	# MSN counts the login server as a "key port" by POSTing to the root of the server with no content.
 	return web.Response(status = 200)
 	
-async def handle_avatar(req: web.Request, small: bool = False) -> web.Response:
+async def handle_avatar(req: web.Request, small: bool = False, msn: bool = False) -> web.Response:
 	uuid = req.match_info['uuid']
-	storage_path: Path = _get_avatar_path(uuid)
+	backend: Backend = req.app['backend']
 
-	if not (storage_path.exists() and storage_path.is_dir()):
-		return await _get_default_avatar(small)
-	else:
-		chosen: Path | None = None
-		for p in storage_path.iterdir():
-			if not p.is_file():
-				continue
-			if not p.name.startswith(uuid):
-				continue
-			has_thumb = "_thumb" in p.stem
-			if has_thumb == small:
-				chosen = p
-				break
+	user = backend.user_service.get(uuid)
+	avatar_md5 = user.avatar if user else None
 
-		if chosen is None:
-			for p in storage_path.iterdir():
-				if p.is_file() and p.name.startswith(uuid):
-					chosen = p
-					break
+	result = get_avatar_data(uuid, avatar_md5, small=small, msn=msn)
+	if result is not None:
+		data, content_type = result
+		return web.Response(status=200, body=data, content_type=content_type)
 
-		if chosen is not None and chosen.exists():
-			try:
-				data = chosen.read_bytes()
-				if not data:
-					chosen = None
-				else:
-					ext = chosen.suffix.lstrip(".").lower() or "png"
-					content_type = f"image/{ext}"
-					return web.Response(status=200, body=data, content_type=content_type)
-			except (PermissionError, IsADirectoryError, OSError):
-				chosen = None
-
-		return await _get_default_avatar(small)
+	return await _get_default_avatar(small)
 
 async def _get_default_avatar(small: bool) -> web.Response:
-	default_url = "https://static.ugnet.gay/svc/userstore/avatar/default-sm.png" if small else "https://static.ugnet.gay/svc/userstore/avatar/default.png"
+	default_url = f"https://{settings.STATIC_HOST}/svc/userstore/avatar/default-sm.png" if small else f"https://{settings.STATIC_HOST}/svc/userstore/avatar/default.png"
 	timeout = ClientTimeout(total=5)
 	try:
 		async with ClientSession(timeout=timeout) as session:
@@ -211,4 +189,4 @@ def render(req: web.Request, tmpl_name: str, ctxt: Optional[Dict[str, Any]] = No
 		content_type = 'text/html'
 	tmpl = req.app['jinja_env'].get_template(tmpl_name)
 	content = tmpl.render(**(ctxt or {}))
-	return web.Response(status=status, content_type=content_type, text=content)
+	return web.Response(status=status, content_type=content_type, text=content)

core/interservice/ctrl.py

@@ -236,6 +236,11 @@ class INSCtrl:
 		self.send_numeric(StatusCode.CircleActionSuccessful, ':CIRCLE {}'.format(ts))
 
 	def _m_alert(self, ts: str, type: str, content: str = '', url: str = '', targets: str = 'all', icon: str = '') -> None:
+		if not self.authenticated:
+			self.send_numeric(Err.NotAuthenticated)
+			self.close()
+			return
+
 		def _quote_circumcision(s: str) -> str:
 			if not s or len(s) < 2:
 				return s
@@ -289,6 +294,11 @@ class INSCtrl:
 
 
 	def _m_user(self, ts: str, action: str, uuid: str, field: str = '', *args: str) -> None:
+		if not self.authenticated:
+			self.send_numeric(Err.NotAuthenticated)
+			self.close()
+			return
+
 		def _parse_value_for_column(col, value_str):
 			py_type = getattr(col.type, 'python_type', None)
 			if py_type is not None:
@@ -336,7 +346,10 @@ class INSCtrl:
 					flag_modified(user_obj, '_front_data')
 			
 			if 'oldaim' in flags_list:
-				user_obj.set_front_data('aim', 'pw_md5', hash.hasher_md5.encode(pw, identifier='AOL Instant Messenger (SM)'))
+				pw_md5_encoded = hash.hasher_md5.encode(pw, identifier='AOL Instant Messenger (SM)')
+				pw_md5_salt = hash.hasher_md5.extract_salt(pw_md5_encoded)
+				user_obj.set_front_data('aim', 'pw_md5', pw_md5_encoded)
+				user_obj.set_front_data('aim', 'pw_md5_v5', hash.hasher_md5.encode_aim5(pw, salt=pw_md5_salt))
 			else:
 				if user_obj._front_data and 'aim' in user_obj._front_data:
 					user_obj._front_data.pop('aim', None)
@@ -407,8 +420,9 @@ class INSCtrl:
 							last_name=last_name,
 							username=username,
 							uin=uin,
-							verified_to_login=True,
+							verified_to_login=False,
 							account_verified=False,
+							alias_active=False,
 							friendly_name=email,
 							groups={},
 							settings={},
@@ -447,6 +461,22 @@ class INSCtrl:
 
 		elif action == 'UPDATE':
 			new_raw_value = ' '.join(args)
+			try:
+				domain_user = self.backend._load_user_record(uuid)
+				if domain_user:
+					for bs in self.backend.util_get_sessions_by_user(domain_user):
+						try:
+							bs.close()
+						except Exception:
+							pass
+			except Exception:
+				traceback.print_exc()
+			if field == 'alias_active':
+				try:
+					self.backend._handle_worklist_notify()
+					self.backend._worklist_notify.clear()
+				except Exception:
+					traceback.print_exc()
 			try:
 				with db.Session() as sess:
 					user_row = sess.query(db.User).filter_by(uuid=uuid).one_or_none()
@@ -469,18 +499,15 @@ class INSCtrl:
 						except Exception as e:
 							self.logger.error(e)
 							self.logger.info("Failed to nuke stale user cache")
-					elif field in ('show_in_dir', 'verified_to_login', 'suspended', 'is_tester', 'is_mvp'):
+					#elif field not in ('account_verified', 'verified_to_login') and not user_row.verified_to_login:
+					#	return self.send_numeric(Err.EmailNotVerified, ts)
+					elif field in ('alias_active') and not user_row.account_verified:
+						return self.send_numeric(Err.EmailNotVerified, ts) 
+					elif field in ('show_in_dir', 'alias_active', 'account_verified' 'verified_to_login', 'suspended', 'is_tester', 'is_mvp'):
 						col = user_row.__table__.columns[field]
 						parsed_val = _parse_value_for_column(col, new_raw_value)
 						setattr(user_row, field, parsed_val)
 						sess.add(user_row)
-						try:
-							domain_user = self.backend._load_user_record(uuid)
-							if domain_user:
-								setattr(domain_user, field, parsed_val)
-						except Exception as e:
-							self.logger.error(e)
-							self.logger.info(f"Failed to update cached user object for {field}")
 					elif field.startswith('profile.'):
 						prof_field = field.split('.', 1)[1]
 						profile = sess.query(db.UserProfile).filter_by(user_id=user_row.id).one_or_none()
@@ -584,26 +611,26 @@ class INSCtrl:
 			except Exception:
 				traceback.print_exc()
 				return self.send_numeric(Err.ServerUnknownError, ts)
+			try:
+				domain_user = self.backend._load_user_record(uuid)
+			except Exception:
+				domain_user = None
 			if field == 'email':
 				try:
-					domain_user = self.backend._load_user_record(uuid)
 					if domain_user:
 						domain_user.email = new_raw_value
 				except Exception as e:
 					self.logger.error(e)
 					self.logger.info("Failed to update cached email")
+			if field == 'alias_active':
+				try:
+					if domain_user:
+						v = new_raw_value.strip().lower()
+						domain_user.alias_active = v in ('1', 'true', 'yes', 'on')
+				except Exception as e:
+					self.logger.error(e)
+					self.logger.info("Failed to update cached alias status")
 			try:
-				domain_user = self.backend._load_user_record(uuid)
-				if domain_user:
-					for bs in self.backend.util_get_sessions_by_user(domain_user):
-						try:
-							bs.close()
-						except Exception:
-							pass
-			except Exception:
-				traceback.print_exc()
-			try:
-				domain_user = self.backend._load_user_record(uuid)
 				if domain_user:
 					try:
 						detail = self.backend._load_detail(domain_user)
@@ -658,6 +685,10 @@ class INSCtrl:
 
 	def _m_allthesessions(self, ts: str, email_filter: str = '') -> None:
 		# TODO: make this a payload command
+		if not self.authenticated:
+			self.send_numeric(Err.NotAuthenticated)
+			self.close()
+			return
 		sessions_info = []
 		for bs in self.backend._sc.iter_sessions():
 			if email_filter and bs.user.email != email_filter:
@@ -667,6 +698,10 @@ class INSCtrl:
 		self.send_reply('ALLTHESESSIONS', ts, *sessions_info)
 
 	def _m_session(self, ts: str, sess_id: str, method: str) -> None:
+		if not self.authenticated:
+			self.send_numeric(Err.NotAuthenticated)
+			self.close()
+			return
 		target = None
 		for bs in self.backend._sc.iter_sessions():
 			if str(id(bs)) == sess_id:
@@ -885,6 +920,7 @@ class Err:
 	EmailTaken = 411
 	RestrictedUseranme = 415
 	RestrictedEmail = 416
+	EmailNotVerified = 419
 	TooManyCharactersUsername = 425
 	TooManyCharactersEmail = 426
 	UserCreationFailed = 427

core/models.py

@@ -12,6 +12,8 @@ class User:
 		'last_name',
 		'uin',
 		'verified_to_login',
+		'account_verified',
+		'alias_active',
 		'status',
 		'detail',
 		'settings',
@@ -23,6 +25,7 @@ class User:
 		'show_in_dir',
 		'evil_permanent',
 		'evil_temporary',
+		'avatar',
 		'profile'
 	)
 	
@@ -34,6 +37,8 @@ class User:
 	last_name: str
 	uin: int
 	verified_to_login: bool
+	account_verified: bool
+	alias_active: bool
 	status: 'UserStatus'
 	detail: Optional['UserDetail']
 	settings: Dict[str, Any]
@@ -45,11 +50,12 @@ class User:
 	show_in_dir: bool
 	evil_permanent: int
 	evil_temporary: int
+	avatar: str
 	profile: 'UserProfile'
 	
 	def __init__(
-		self, id: int, uuid: str, email: str, username: str, first_name: str, last_name: str, uin: int, verified_to_login: bool, status: 'UserStatus',
-		settings: Dict[str, Any], date_created: datetime, date_login: datetime, suspended: bool, is_tester: bool, is_mvp: bool, show_in_dir: bool, evil_permanent: int, evil_temporary: int, profile: 'UserProfile'
+		self, id: int, uuid: str, email: str, username: str, first_name: str, last_name: str, uin: int, verified_to_login: bool, account_verified: bool, alias_active: bool, status: 'UserStatus',
+		settings: Dict[str, Any], date_created: datetime, date_login: datetime, suspended: bool, is_tester: bool, is_mvp: bool, show_in_dir: bool, evil_permanent: int, evil_temporary: int, avatar: str, profile: 'UserProfile'
 	) -> None:
 		self.id = id
 		self.uuid = uuid
@@ -59,6 +65,8 @@ class User:
 		self.last_name = last_name
 		self.uin = uin
 		self.verified_to_login = verified_to_login
+		self.account_verified = account_verified
+		self.alias_active = alias_active
 		# `status`: true status of user
 		self.status = status
 		self.detail = None
@@ -71,6 +79,7 @@ class User:
 		self.show_in_dir = show_in_dir
 		self.evil_permanent = evil_permanent
 		self.evil_temporary = evil_temporary
+		self.avatar = avatar
 		self.profile = profile
 
 class Contact:

core/tmpl/misc/miscconns.html

@@ -1,7 +1,7 @@
 <html>
 <body>
 <h1>Here be dragons!</h1>
-<p>You appear to be lost, go <a href="https://crosstalk.im">here</a>.</p>
-<i><small>Powered by Azul <b>{{ settings.VERSION }}-{% if settings.DEBUG %}DEBUG{% else %}PROD{% endif %}</b></small></i>
+<p>You appear to be lost, go <a href="https://{{ settings.WEBSITE_HOST }}">here</a>.</p>
+<i><small>Powered by Azul <b>{{ settings.VERSION }} ({% if settings.DEBUG %}DEBUG{% else %}PROD{% endif %})</b></small></i>
 </body>
 </html>

core/user.py

@@ -8,6 +8,7 @@ import json, traceback
 
 from util.hash import gen_salt, hasher, hasher_md5, hasher_md5crypt
 from util import misc
+from util.avatar import store_avatar_file, get_img_type
 
 from .conn import Conn
 from .db import (
@@ -99,6 +100,14 @@ class UserService:
 
 			return hasher_md5.extract_hash(dbuser.get_front_data('aim', 'pw_md5') or '')
 
+	def aim_get_md5_password_bucp2(self, screen_name: str) -> Optional[bytes]:
+		with self._conn.session() as sess:
+			dbuser = sess.query(DBUser).filter(func.lower(DBUser.username) == screen_name.lower()).one_or_none()
+			if dbuser is None:
+				return None
+
+			return hasher_md5.extract_hash(dbuser.get_front_data('aim', 'pw_md5_v5') or '')
+
 	def aim_get_md5_salt(self, screen_name: str) -> Optional[str]:
 		with self._conn.session() as sess:
 			dbuser = sess.query(DBUser).filter(func.lower(DBUser.username) == screen_name.lower()).one_or_none()
@@ -109,6 +118,7 @@ class UserService:
 
 		return hasher.extract_salt(pw_md5) if pw_md5 else None
 
+
 	def msim_get_sha1_password(self, email: str) -> Optional[bytes]:
 		with self._conn.session() as sess:
 			dbuser = sess.query(DBUser).filter(DBUser.email == email).one_or_none()
@@ -146,7 +156,30 @@ class UserService:
 		if uuid not in self._cache_by_uuid:
 			self._cache_by_uuid[uuid] = self._get_uncached(uuid)
 		return self._cache_by_uuid[uuid]
-	
+
+	def get_user_email(self, user: User) -> str:
+		return user.email.lower()
+		
+	def get_display_email(self, user: User) -> str:
+		if user.alias_active:
+			return '{}@crosstalk.im'.format(user.username).lower()
+		return user.email.lower()
+
+	def lookup_uuid_by_email(self, email: str) -> Optional[str]:
+		uuid = self.get_uuid(email)
+		if uuid is not None:
+			return uuid
+		lower = email.lower()
+		if lower.endswith('@crosstalk.im'):
+			username = lower[:-len('@crosstalk.im')]
+			uuid = self.get_uuid_username(username)
+			if uuid is not None:
+				return uuid
+		else:
+			local = email.split('@')[0]
+			uuid = self.get_uuid_username(local)
+		return uuid
+
 	def _get_uncached(self, uuid: str) -> Optional[User]:
 		with self._conn.session() as sess:
 			dbuser = sess.query(DBUser).options(joinedload(DBUser.profile)).filter(DBUser.uuid == uuid).one_or_none()
@@ -172,11 +205,23 @@ class UserService:
 			status = UserStatus(dbuser.friendly_name or dbuser.email)
 			return User(
 				dbuser.id, dbuser.uuid, dbuser.email, dbuser.username, dbuser.first_name, dbuser.last_name, dbuser.uin,
-				dbuser.verified_to_login, status, dbuser.settings, dbuser.date_created, dbuser.date_login, dbuser.suspended,
-				dbuser.is_tester, dbuser.is_mvp, dbuser.show_in_dir, dbuser.evil_permanent, dbuser.evil_temporary,
+				dbuser.verified_to_login, dbuser.account_verified, dbuser.alias_active, status, dbuser.settings, dbuser.date_created, dbuser.date_login, dbuser.suspended,
+				dbuser.is_tester, dbuser.is_mvp, dbuser.show_in_dir, dbuser.evil_permanent, dbuser.evil_temporary, dbuser.avatar,
 				profile=user_profile
 			)
-	
+
+	def save_avatar(self, user: User, data: bytes, ext: str) -> str:
+		md5 = store_avatar_file(user.uuid, data, ext)
+
+		with self._conn.session() as sess:
+			sess.query(DBUser).filter(DBUser.uuid == user.uuid).update({'avatar': md5})
+
+		self._cache_by_uuid.pop(user.uuid, None)
+
+		user.avatar = md5
+		return md5
+
+
 	def get_detail(self, uuid: str) -> Optional[UserDetail]:
 		with self._conn.session() as sess:
 			dbuser = sess.query(DBUser).filter(DBUser.uuid == uuid).one_or_none()

deps.txt

@@ -6,8 +6,8 @@ pillow
 MarkupSafe
 HLL==1.3.1
 funcli
-git+https://git.hiden.cc/CrossTalk/brutus.git
-git+https://git.hiden.cc/CrossTalk/ins-client.git
+git+https://git.ugnet.gay/CrossTalk/brutus.git
+git+https://git.ugnet.gay/CrossTalk/ins-client.git
 pytz
 cryptography
 arc4

docs.txt

@@ -10,8 +10,8 @@ http://web.archive.org/web/20100924153734/http://www.ycoderscookbook.com/tutoria
 https://github.com/afallenhope/web-ymsg
 http://web.archive.org/web/20090623064155/carbonize.co.uk/ymsg16.html
 http://web.archive.org/web/20040803082402/http://www.carbonize.co.uk:80/Tutorials/Vb/ycht.php
-https://wiki.nina.chat/wiki/Protocols/YMSG
-https://wiki.nina.chat/wiki/Protocols/YMSG/Packet_Structure
+https://ninawiki.preloading.dev/wiki/Protocols/YMSG
+https://ninawiki.preloading.dev/wiki/Protocols/YMSG/Packet_Structure
 YMSG8:
 http://web.archive.org/web/20010801212305/http://www.venkydude.com:80/articles/yahoo.htm 1
 YMSG9:
@@ -23,7 +23,7 @@ http://web.archive.org/web/20091229030156/http://www.venkydude.com/articles/yaho
 
 MSNP:
 https://github.com/Richie-Zhang/Instant_Messenger/blob/master/documents/Internet%20Draft%20MSNP%201.0
-https://wiki.nina.chat/wiki/Protocols/MSNP/Documents
+https://ninawiki.preloading.dev/wiki/Protocols/MSNP/Documents
 http://www.hypothetic.org/docs/msn/
 https://protogined.wordpress.com/msnp2/
 https://protogined.wordpress.com/msnp3/
@@ -71,7 +71,7 @@ http://web.archive.org/web/20061116183306/http://www.therisenrealm.com/scripts/b
 http://plaza.ufl.edu/dmitrid/perl/
 https://github.com/wiseman/claim
 https://metacpan.org/pod/Net::AOLIM
-https://wiki.nina.chat/wiki/Protocols/TOC/1.0
+https://ninawiki.preloading.dev/wiki/Protocols/TOC/1.0
 
 OSCAR:
 https://web.archive.org/web/20060819080319/http://www.oilcan.org/oscar/

front/irc/ctrl.py

@@ -169,18 +169,12 @@ class IRCCtrl:
 	def _m_motd(self) -> None:
 		self.send_numeric(RPL.MOTDStart, ": ----- CrossTalk (BETA) -----")
 		self.send_numeric(RPL.MOTD, ": - Welcome to CrossTalk! This frontend is undergoing a massive overhaul, so expect things to be iffy for now.")
-		self.send_numeric(RPL.MOTD, ": - We support MSNP, YMSG, and OSCAR as well, though as of right now, only MSNP and YMSG are in a usable state.")
+		self.send_numeric(RPL.MOTD, ": - We support MSNP, YMSG, and OSCAR as well.")
 		self.send_numeric(RPL.MOTD, ": - Development is going at a faster pace, so this won't stay the case for long. Stay tuned!")
-		#if settings.PRIVACY_POLICY_UPDATE_NOTIF:
-		#	self.send_numeric(RPL.MOTD, ": - NOTE: We have updated our Privacy Policy. Go here to take a look: https://crosstalk.im/ppolicy")
-		#if settings.TOS_UPDATE_NOTIF:
-		#	self.send_numeric(RPL.MOTD, ": - NOTE: We have updated our Terms of Service. You must read and agree to these to continue using CrossTalk. Go here to take a look: https://crosstalk.im/tos")
-		#if settings.REROUTE_UPDATE_NOTIF:
-		#	settings.send_numeric(RPL.MOTD, ": - NOTE for Yahoo! and MSN users: We have updated Reroute. Please download a new copy and replace your existing DLL with it. You can find that here: https://storage.ugnet.gay/crosstalk-dist/client/all/patching/reroute/reroute.dll")
 		self.send_numeric(RPL.MOTDEnd, ": - End of the MOTD.")
 
 	def _m_version(self) -> None:
-		self.send_numeric(RPL.Version, ": - Azul (BETA) - IRC Frontend")
+		self.send_numeric(RPL.Version, ": - AZUL (BETA) - IRC")
 	
 	def _m_quit(self, reason: Optional[str]) -> None:
 		self.close()

front/msim/http.py

@@ -8,4 +8,4 @@ def register(app: web.Application) -> None:
 
 
 async def handle_client_advert(request: web.Request) -> web.Response:
-	return web.HTTPFound(f'http://ctsvcs.advertising.ugnet.gay/ads/banner-sq')
+	return web.HTTPFound(f'http://{settings.AD_HOST}y/ads/banner-sq')

front/msn/http/abservice.py

@@ -1389,12 +1389,12 @@ def ab_BreakConnection(req: web.Request, header: Any, action: Any, bs: BackendSe
 	})
 
 async def _dispatch_circle_created(backend: Backend, user: models.User, circle: models.Circle) -> None:
-	await asyncio.sleep(0.0125)
+	await asyncio.sleep(0.04)
 	for sess in backend.util_get_sessions_by_user(user):
 		sess.evt.on_circle_created(circle)
 
 async def _dispatch_circle_invite_status(backend: Backend, user: models.User, circle: models.Circle, accepted: bool) -> None:
-	await asyncio.sleep(0.0125)
+	await asyncio.sleep(0.04)
 	if accepted:
 		for sess in backend.util_get_sessions_by_user(user):
 			sess.evt.on_accepted_circle_invite(circle)
@@ -1405,7 +1405,7 @@ async def _dispatch_circle_invite_status(backend: Backend, user: models.User, ci
 			sess.evt.on_declined_chat_invite(chat, circle = True)
 
 async def _dispatch_circle_left(backend: Backend, user: models.User, circle: models.Circle) -> None:
-	await asyncio.sleep(0.0125)
+	await asyncio.sleep(0.04)
 	for sess in backend.util_get_sessions_by_user(user):
 		sess.evt.on_left_circle(circle)
 

front/msn/http/other.py

@@ -12,6 +12,7 @@ from aiohttp import web
 
 from core import models
 from core.backend import Backend, BackendSession
+from util.avatar import get_img_type, find_avatar_file
 from ..misc import gen_mail_data, format_oim, cid_format, puid_format
 from .util import find_element, get_tag_localname, render, preprocess_soap, unknown_soap, bool_to_str
 
@@ -60,7 +61,7 @@ def register(app: web.Application) -> None:
 	app.router.add_post('/ppsecure/sha1auth.srf', handle_sha1auth)
 	app.router.add_post('/rsi/rsi.asmx', handle_rsi)
 	app.router.add_post('/OimWS/oim.asmx', handle_oim)
-	app.router.add_get('/svcs/msn-video-feeds/', handle_videofeed) # adding an ending back-slash is required, or else it 404s. ????
+	app.router.add_get('/svcs/msn-video-feeds/', handle_videofeed)
 	app.router.add_get('/svcs/feed', lambda req: handle_videofeed(req, get_videos = True))
 
 	# MSN >= 14
@@ -102,29 +103,26 @@ async def handle_storageservice(req: web.Request) -> web.Response:
 	if action_str == 'GetProfile':
 		roaming_info = backend.user_service.get_roaming_info(user)
 		assert roaming_info is not None
-		
-		storage_path = _get_storage_path(user.uuid)
-		files = None
-		if storage_path.exists() and storage_path.is_dir():
-			files = [x for x in storage_path.iterdir() if '_thumb' not in x.stem]
-		
-		mime = None
-		image_size = 0
-		image_thumb_size = 0
-		
-		if files:
-			ext = files[0].suffix
-			mime = ext[1:]
-			image_path = storage_path / '{}{}'.format(user.uuid, ext)
-			image_thumb_path = storage_path / '{}_thumb{}'.format(user.uuid, ext)
-			
-			if image_path.exists():
-				image_size = image_path.stat().st_size
-			
-			if image_thumb_path.exists():
-				image_thumb_size = image_thumb_path.stat().st_size
 
-		
+		# Resolve the active avatar using the md5 stored in users.avatar.
+		# find_avatar_file returns None if no avatar has been uploaded yet.
+		avatar_md5 = user.avatar
+		mime		= None
+		image_size  = 0
+		image_thumb_size = 0
+
+		if avatar_md5:
+			full_file  = find_avatar_file(user.uuid, avatar_md5, small=False)
+			thumb_file = find_avatar_file(user.uuid, avatar_md5, small=True)
+
+			if full_file and full_file.is_file():
+				ext  = full_file.suffix.lstrip('.').lower()
+				mime = ext
+				image_size = full_file.stat().st_size
+
+			if thumb_file and thumb_file.is_file():
+				image_thumb_size = thumb_file.stat().st_size
+
 		return render(req, 'msn:storageservice/GetProfileResponse.xml', {
 			'storage_ns': storage_ns,
 			'cachekey': cachekey,
@@ -228,8 +226,9 @@ async def handle_storageservice(req: web.Request) -> web.Response:
 			'pptoken1': token,
 			'user': user,
 		})
-	if action_str in ('CreateDocument','UpdateDocument'):
-		return handle_document(req, action, ('Update' if action_str == 'UpdateDocument' else 'Create'), storage_ns, user, cid, token)
+	if action_str in ('CreateDocument', 'UpdateDocument'):
+		op = 'Update' if action_str == 'UpdateDocument' else 'Create'
+		return handle_document(req, action, op, storage_ns, user, cid, token, backend)
 	if action_str == 'CreateRelationships':
 		# TODO: CreateRelationships
 		return render(req, 'msn:storageservice/CreateRelationshipsResponse.xml', {
@@ -244,65 +243,57 @@ async def handle_storageservice(req: web.Request) -> web.Response:
 		return unknown_soap(req, header, action, expected = True)
 	return unknown_soap(req, header, action)
 
-def handle_document(req: web.Request, action: Any, type: str, storage_ns: str, user: models.User, cid: str, token: str) -> web.Response:
-	from PIL import Image
-	
-	# get image data
-	#name = find_element(action, 'Name')
+
+def handle_document(
+	req: web.Request,
+	action: Any,
+	op_type: str,		  # 'Create' or 'Update'
+	storage_ns: str,
+	user: models.User,
+	cid: str,
+	token: str,
+	backend: Backend,
+) -> web.Response:
 	streamtype = find_element(action, 'DocumentStreamType')
-	
+
 	if streamtype == 'UserTileStatic':
-		mime = find_element(action, 'MimeType')
-		#mime = None
-		data = find_element(action, 'Data')
-		data = base64.b64decode(data)
-		
-		# WLM sends either `png` or `image/png` as the MIME type no matter what type of file is sent over. Guess image type from header
-		
-		# TODO: BMPs
-		if data[:6] in (b'GIF87a',b'GIF89a'):
-			mime = 'gif'
-		elif data[:2] == b'\xff\xd8':
-			mime = 'jpeg'
-		elif data[:8] == b'\x89PNG\x0d\x0a\x1a\x0a':
-			mime = 'png'
-		
-		if mime is not None:
+		raw_b64 = find_element(action, 'Data')
+		if raw_b64:
 			try:
-				image = Image.open(BytesIO(data))
-			except:
-				return web.HTTPInternalServerError(text = '')
-			
-			# store display picture as file
-			path = _get_storage_path(user.uuid)
-			path.mkdir(exist_ok = True, parents = True)
-			
-			for old_file in path.glob(f'{user.uuid}.*'):
-				if old_file.is_file():
-					old_file.unlink()
-			for old_thumb in path.glob(f'{user.uuid}_thumb.*'):
-				if old_thumb.is_file():
-					old_thumb.unlink()
-			
-			image_path = path / '{uuid}.{mime}'.format(uuid = user.uuid, mime = mime)
-			
-			image_path.write_bytes(data)
-			
-			thumb = image.resize((21, 21))
-			
-			thumb_path = path / '{uuid}_thumb.png'.format(uuid = user.uuid)
-			thumb.save(str(thumb_path))
-	
-	return render(req, 'msn:storageservice/{}DocumentResponse.xml'.format(type), {
+				data = base64.b64decode(raw_b64)
+			except Exception:
+				return web.HTTPInternalServerError(text='')
+
+			err = _store_avatar_from_bytes(backend, user, data)
+			if err is not None:
+				return err
+
+	return render(req, 'msn:storageservice/{}DocumentResponse.xml'.format(op_type), {
 		'storage_ns': storage_ns,
 		'cid': cid,
 		'pptoken1': token,
 		'user': user,
 	})
 
+
+def _store_avatar_from_bytes(
+	backend: Backend,
+	user: models.User,
+	data: bytes,
+) -> Optional[web.Response]:
+	ext = get_img_type(data)
+	if ext is None:
+		return web.HTTPInternalServerError(text='AZUL: Unsupported image format')
+
+	try:
+		backend.user_service.save_avatar(user, data, ext)
+	except Exception:
+		return web.HTTPInternalServerError(text='')
+
+	return None
+
+
 async def handle_sha1auth(req: web.Request) -> web.Response:
-	# We have no use for any of the actual tokens sent here right now (this is primarily for WLM 8's MSN Today function),
-	# so just redirect to the URL specified by `ru`
 	post = await req.post()
 	
 	token_data = post.get('token')
@@ -388,7 +379,9 @@ async def handle_oim(req: web.Request) -> web.Response:
 	
 	recipient_uuid = backend.util_get_uuid_from_email(recipient)
 	
-	if email != user.email or recipient_uuid is None or not _is_on_al(recipient_uuid, backend, user, detail):
+	user_display_email = backend.util_get_display_email(user)
+	alias_email = '{}@crosstalk.im'.format(user.username).lower()
+	if email.lower() not in (user.email.lower(), alias_email, user_display_email.lower()) or recipient_uuid is None or not _is_on_al(recipient_uuid, backend, user, detail):
 		return render(req, 'msn:oim/Fault.unavailable.xml', {
 			'owsns': owsns,
 		}, status = 500)
@@ -588,13 +581,13 @@ async def handle_gamebrowser(req: web.Request) -> web.Response:
 	})
 
 async def handle_portal(req: web.Request) -> web.Response:
-	return web.HTTPFound(f'http://today.msgrsvcs.ctsrv.gay/start/msn')
+	return web.HTTPFound(f'http://{settings.TODAY_HOSTs}/start/msn')
 
 async def handle_textadredir(req: web.Request) -> web.Response:
-	return web.HTTPFound(f'http://ctsvcs.advertising.ugnet.gay/ads/txt')
+	return web.HTTPFound(f'http://{settings.AD_HOST}/ads/txt')
 
 async def handle_banneradredir(req: web.Request) -> web.Response:
-	return web.HTTPFound(f'http://ctsvcs.advertising.ugnet.gay/ads/banner')
+	return web.HTTPFound(f'http://{settings.AD_HOST}/ads/banner')
 
 async def handle_msn_redirect(req: web.Request) -> web.Response:
 	i = req.match_info['i']
@@ -844,9 +837,6 @@ async def handle_rst(req: web.Request, rst2: bool = False) -> web.Response:
 		'timez': util.misc.date_format(datetime.utcnow()),
 	})
 
-def _get_storage_path(uuid: str) -> Path:
-	return Path('storage/dp') / uuid[0:1] / uuid[0:2]
-
 async def handle_videofeed(req: web.Request, get_videos: bool = False) -> web.Response:
 	if not get_videos:
 		return render(req, 'msn:svcs/videofeedroot.xml')
@@ -907,6 +897,9 @@ def _extract_pp_credentials(auth_str: str) -> Optional[Tuple[str, str]]:
 def _login(req: web.Request, email: str, pwd: str, binary_secret: bool = False, lifetime: int = 30) -> Optional[Tuple[str, datetime, Optional[str]]]:
 	backend: Backend = req.app['backend']
 	uuid = backend.user_service.login(email, pwd)
+	if uuid is None and email.lower().endswith('@crosstalk.im'):
+		username = email.lower()[:-len('@crosstalk.im')]
+		uuid = backend.user_service.login_with_username(username, pwd)
 	if uuid is None: return None
 	bsecret = None
 	if binary_secret:
@@ -918,4 +911,4 @@ def _contact_is_favorite(user_detail: models.UserDetail, ctc: models.Contact) ->
 	for group in ctc._groups.copy():
 		if group.id not in groups: continue
 		if groups[group.id].is_favorite: return True
-	return False
+	return False

front/msn/http/tmpl/config/Config.PSA.xml

@@ -1790,8 +1790,8 @@
 		</App>
 		<App DomainId="70">
 			<AppId>1140906031</AppId>
-			<ThumbNailUrl>https://cts.storage.ugnet.gay/static/social/ugnetthumb.gif</ThumbNailUrl>
-			<MiniPhotoUrl>https://cts.storage.ugnet.gay/static/social/mini.gif</MiniPhotoUrl>
+			<ThumbNailUrl>https://{{ settings.USERSTORAGE_HOST }}/static/social/ugnetthumb.gif</ThumbNailUrl>
+			<MiniPhotoUrl>https://{{ settings.USERSTORAGE_HOST }}/static/social/mini.gif</MiniPhotoUrl>
 			<Domain>^https?://.*\.hiden\.cc/.*$</Domain>
 			<EmailDomain>mail.ugnet.gay</EmailDomain>
 			<WebDomain>ugnet.gay</WebDomain>

front/msn/http/tmpl/config/MsgrConfig.msn.xml

@@ -63,19 +63,19 @@
 	</TabConfig>
 	<AbchCfg>
 		<abchconfig>
-			<url>https://ctsvcs.addressbook.ugnet.gay/abservice.asmx</url>
+			<url>https://{{ settings.ADDRESSBOOK_HOST }}/abservice.asmx</url>
 		</abchconfig>
 	</AbchCfg>
 	<LocalizedConfig Market="en-US">
 		<AdMainConfig>
 			<TextAdRefresh>1</TextAdRefresh>
-			<TextAdServer>http://ctsvcs.advertising.ugnet.gay/ads/txt</TextAdServer>
-			<AdBanner20URL Refresh="300">http://ctsvcs.advertising.ugnet.gay/ads/banner</AdBanner20URL>
+			<TextAdServer>http://{{ settings.AD_HOST }}/ads/txt</TextAdServer>
+			<AdBanner20URL Refresh="300">http://{{ settings.AD_HOST }}/ads/banner</AdBanner20URL>
 		</AdMainConfig>
 		<AppDirConfig>
-			<AppDirPageURL>http://mactivities.msgrsvcs.ctsrv.gay/AppDirectory/Directory.aspx?L=en-us</AppDirPageURL>
-			<AppDirSeviceURL>http://mactivities.msgrsvcs.ctsrv.gay/AppDirectory/AppDirectory.asmx</AppDirSeviceURL>
-			<AppDirVersionURL>http://mactivities.msgrsvcs.ctsrv.gay/AppDirectory/GetAppdirVersion.aspx</AppDirVersionURL>
+			<AppDirPageURL>http://{{ settings.APPDIR_HOST }}/AppDirectory/Directory.aspx?L=en-us</AppDirPageURL>
+			<AppDirSeviceURL>http://{{ settings.APPDIR_HOST }}/AppDirectory/AppDirectory.asmx</AppDirSeviceURL>
+			<AppDirVersionURL>http://{{ settings.APPDIR_HOST }}/AppDirectory/GetAppdirVersion.aspx</AppDirVersionURL>
 		</AppDirConfig>
 		<MSNSearch>
 			<DesktopInstallURL>https://searx.ugnet.gay/searxng/search?q=$QUERY$&amp;ia=web</DesktopInstallURL>
@@ -88,15 +88,15 @@
 			<SharedSearchURL2>https://searx.ugnet.gay/searxng/search?q=$QUERY$&amp;ia=web</SharedSearchURL2>
 		</MSNSearch>
 		<MsnTodayConfig>
-			<MsnTodayURL>http://today.msgrsvcs.ctsrv.gay/start?msn=1</MsnTodayURL>
+			<MsnTodayURL>http://{{ settings.TODAY_HOST }}/start?msn=1</MsnTodayURL>
 		</MsnTodayConfig>
 		<MusicIntegration URL="https://www.last.fm/search/tracks?q=$ARTIST$+$TITLE$"/>
 		<RL>
-			<ViewProfileURL>https://crosstalk.im/account/settings</ViewProfileURL>
+			<ViewProfileURL>https://{{ settings.WEBSITE_HOST }}/account/settings</ViewProfileURL>
 		</RL>
 		<TermsOfUse>
 			<TermsOfUseSID>956</TermsOfUseSID>
-			<TermsOfUseURL>https://crosstalk.im/tos</TermsOfUseURL>
+			<TermsOfUseURL>https://{{ settings.WEBSITE_HOST }}/tos</TermsOfUseURL>
 		</TermsOfUse>
 	</LocalizedConfig>
 </MsgrConfig>

front/msn/http/tmpl/config/MsgrConfig.wlm.14.xml

@@ -102,30 +102,30 @@
 			<Device VID="0x045e" PID="0x0722"/>
 			<Device VID="0x041e" PID="0x4080"/>
 		</USBDevices>
-		<CirclesWeb General="https://crosstalk.im/circles/$CIRCLECID$/settings/" General-SiteId="73625" Management="https://crosstalk.im/circles/$CIRCLECID$/settings/" Management-SiteId="73625" Membership="https://crosstalk.im/circles/$CIRCLECID$/settings/" Membership-SiteId="73625" Leave="https://crosstalk.im/circles/$CIRCLECID$/settings/" Leave-SiteId="73625" Disc="https://discord.gg/dumJwXTPxX" />
+		<CirclesWeb General="https://{{ settings.WEBSITE_HOST }}/circles/$CIRCLECID$/settings/" General-SiteId="73625" Management="https://{{ settings.WEBSITE_HOST }}/circles/$CIRCLECID$/settings/" Management-SiteId="73625" Membership="https://{{ settings.WEBSITE_HOST }}/circles/$CIRCLECID$/settings/" Membership-SiteId="73625" Leave="https://{{ settings.WEBSITE_HOST }}/circles/$CIRCLECID$/settings/" Leave-SiteId="73625" Disc="https://{{ settings.WEBSITE_HOST }}" />
 		<AnonymousIMSettings>
 			<SettingsPage URL="http://settings.messenger.live.com/applications/WebSettings.aspx" SiteID="76215" />
 		</AnonymousIMSettings>
 	</Simple>
 	<AbchCfg>
 		<abchconfig>
-			<url>https://ctsvcs.addressbook.ugnet.gay/abservice.asmx</url>
+			<url>https://{{ settings.ADDRESSBOOK_HOST }}/abservice.asmx</url>
 		</abchconfig>
 	</AbchCfg>
 	<LocalizedConfig Market="en-US">
 		<AdMainConfig>
 			<TextAdRotation>420</TextAdRotation>
 			<TextAdRefresh>1</TextAdRefresh>
-			<TextAdServer>http://ctsvcs.advertising.ugnet.gay/ads/txt</TextAdServer>
-			<AdBanner20URL Refresh="300">http://ctsvcs.advertising.ugnet.gay/ads/banner</AdBanner20URL>
+			<TextAdServer>http://{{ settings.AD_HOST }}/ads/txt</TextAdServer>
+			<AdBanner20URL Refresh="300">http://{{ settings.AD_HOST }}/ads/banner</AdBanner20URL>
 			<StdAd AdVisible="60" VideoVisible="120" VideoDldIntvl="0">
-				<VideoFeeds FeedsListURL="http://ctsvcs.advertising.ugnet.gay/svcs/msn-video-feeds/"/>
+				<VideoFeeds FeedsListURL="http://{{ settings.AD_HOST }}/svcs/msn-video-feeds/"/>
 			</StdAd>
 		</AdMainConfig>
 		<AppDirConfig>
-			<AppDirPageURL>http://mactivities.msgrsvcs.ctsrv.gay/AppDirectory/Directory.aspx?L=en-us</AppDirPageURL>
-			<AppDirSeviceURL>http://mactivities.msgrsvcs.ctsrv.gay/AppDirectory/AppDirectory.asmx</AppDirSeviceURL>
-			<AppDirVersionURL>http://mactivities.msgrsvcs.ctsrv.gay/AppDirectory/GetAppdirVersion.aspx</AppDirVersionURL>
+			<AppDirPageURL>http://{{ settings.APPDIR_HOST }}/AppDirectory/Directory.aspx?L=en-us</AppDirPageURL>
+			<AppDirSeviceURL>http://{{ settings.APPDIR_HOST }}/AppDirectory/AppDirectory.asmx</AppDirSeviceURL>
+			<AppDirVersionURL>http://{{ settings.APPDIR_HOST }}/AppDirectory/GetAppdirVersion.aspx</AppDirVersionURL>
 		</AppDirConfig>
 		<Contacts>
 			<ABPendingRequests>
@@ -136,7 +136,7 @@
 				<urlcid>http://cid-%1!16.16I64X!.profile.live.com/invites?partner=Messenger</urlcid>
 			</ABPendingRequests>
 			<EditMyLiveContactProfile>
-				<url>http://social.ugnet.gay/profile.aspx?action=edit&amp;mode=activecontacts&amp;mkt=en-us&amp;partner=Messenger</url>
+				<url>http://{{ settings.SOCIAL_HOST }}/profile.aspx?action=edit&amp;mode=activecontacts&amp;mkt=en-us&amp;partner=Messenger</url>
 				<SiteId>73625</SiteId>
 			</EditMyLiveContactProfile>
 			<EditMyProfile>
@@ -153,8 +153,8 @@
 				<SiteId>73625</SiteId>
 			</MessengerInvite>
 			<Profile>
-				<url>http://social.ugnet.gay/Profile.aspx?partner=Messenger&amp;mkt=en-us&amp;mem=%ls</url>
-				<urlcid>http://social.ugnet.gay/Profile.aspx?partner=Messenger&amp;mkt=en-us&amp;cid=%ls</urlcid>
+				<url>http://{{ settings.SOCIAL_HOST }}/Profile.aspx?partner=Messenger&amp;mkt=en-us&amp;mem=%ls</url>
+				<urlcid>http://{{ settings.SOCIAL_HOST }}/Profile.aspx?partner=Messenger&amp;mkt=en-us&amp;cid=%ls</urlcid>
 				<urlcid2>http://cid-%1!16.16I64X!.profile.live.com</urlcid2>
 				<SiteId>73625</SiteId>
 			</Profile>
@@ -167,22 +167,22 @@
 		</Contacts>
 		<ContactCard>
 			<AddToMySpaceSiteId>45930</AddToMySpaceSiteId>
-			<AddToMySpaceUrl>http://social.ugnet.gay/?addblogentry=true&amp;mkt=nl-nl</AddToMySpaceUrl>
+			<AddToMySpaceUrl>http://{{ settings.SOCIAL_HOST }}/?addblogentry=true&amp;mkt=nl-nl</AddToMySpaceUrl>
 			<ContactCardDisabled>false</ContactCardDisabled>
 			<CreateSpaceSiteId>45930</CreateSpaceSiteId>
-			<CreateSpaceUrl>http://social.ugnet.gay/signup.aspx?mkt=nl-nl</CreateSpaceUrl>
+			<CreateSpaceUrl>http://{{ settings.SOCIAL_HOST }}/signup.aspx?mkt=nl-nl</CreateSpaceUrl>
 			<GetItemVerUrl>http://ctas.storage.ugnet.gay/storageservice/schematizedstore.asmx</GetItemVerUrl>
-			<GetXmlFeedUrl>http://social.ugnet.gay/contactcard/contactcardservice.asmx</GetXmlFeedUrl>
+			<GetXmlFeedUrl>http://{{ settings.SOCIAL_HOST }}/contactcard/contactcardservice.asmx</GetXmlFeedUrl>
 			<MySpaceSiteId>45930</MySpaceSiteId>
-			<MySpaceUrl>http://social.ugnet.gay/homepage.aspx</MySpaceUrl>
+			<MySpaceUrl>http://{{ settings.SOCIAL_HOST }}/homepage.aspx</MySpaceUrl>
 			<SpaceBaseSiteId>45930</SpaceBaseSiteId>
-			<SpaceBaseUrl>http://social.ugnet.gay</SpaceBaseUrl>
+			<SpaceBaseUrl>http://{{ settings.SOCIAL_HOST }}</SpaceBaseUrl>
 			<SpaceIntegrationEnabled>true</SpaceIntegrationEnabled>
 		</ContactCard>
-		<FlashUpgradeURL>http://storage.ugnet.gay/crosstalk-dist/client/all/tools/flashplayer_10_3r183_48_winax.exe</FlashUpgradeURL>
+		<FlashUpgradeURL>http://{{ settings.STORAGE_HOST }}/crosstalk-dist/client/all/tools/flashplayer_10_3r183_48_winax.exe</FlashUpgradeURL>
 		<GLinkMarket v="en_us" />
 		<LiveFolders UpsellMessageId="1" />
-		<MessengerBlogURL>https://crosstalk.im/blog</MessengerBlogURL>
+		<MessengerBlogURL>https://{{ settings.WEBSITE_HOST }}/blog</MessengerBlogURL>
 		<MSNSearch>
 			<DesktopInstallURL>https://searx.ugnet.gay/searxng/search?q=$QUERY$&amp;ia=web</DesktopInstallURL>
 			<ImagesURL>https://searx.ugnet.gay/searxng/search?q=$QUERY$&amp;ia=images</ImagesURL>
@@ -197,7 +197,7 @@
 		</MSNSearch>
 		<MsnTodayConfig>
 			<MsnTodaySiteID>6528</MsnTodaySiteID>
-			<MsnTodayURL>http://today.msgrsvcs.ctsrv.gay/start?wlm=1</MsnTodayURL>
+			<MsnTodayURL>http://{{ settings.TODAY_HOST }}/start?wlm=1</MsnTodayURL>
 			<Flags>2</Flags>
 		</MsnTodayConfig>
 		<MusicIntegration URL="https://www.last.fm/search/tracks?q=$ARTIST$+$TITLE$"/>
@@ -207,7 +207,7 @@
 		<PSTNOut Hidden="0" />
 		<ReportAbuse>
 			<ReportAbuseSID />
-			<ReportAbuseURL>https://crosstalk.im/report</ReportAbuseURL>
+			<ReportAbuseURL>https://{{ settings.WEBSITE_HOST }}/report</ReportAbuseURL>
 		</ReportAbuse>
 		<RL>
 			<ViewProfileURL>http://g.msn.com/5meen_us/106?%1&amp;Plcid=%2!hs!&amp;%3&amp;Country=%4!hs!&amp;BrandID=%5&amp;passport=%6</ViewProfileURL>
@@ -215,7 +215,7 @@
 		<SDrive Enabled="true" />
 		<TermsOfUse>
 			<TermsOfUseSID>956</TermsOfUseSID>
-			<TermsOfUseURL>https://crosstalk.im/tos</TermsOfUseURL>
+			<TermsOfUseURL>https://{{ settings.WEBSITE_HOST }}/tos</TermsOfUseURL>
 		</TermsOfUse>
 		<VoiceClip Hidden="0" />
 		<VoiceVideo>
@@ -230,7 +230,7 @@
 			<slots>
 				{% for tab in tabs %}
 					<URL id="{{ tab.id }}">
-						https://ctsvcs.advertising.ugnet.gay/svcs/tab={{ tab.id }}
+						https://{{ settings.AD_HOST }}/svcs/tab={{ tab.id }}
 					</URL>
 				{% endfor %}
 				</slots>

front/msn/http/tmpl/config/MsgrConfig.wlm.16.xml

@@ -238,7 +238,7 @@
 		<VoiceVideo>
 			<Log PeerLogDisable="1" />
 		</VoiceVideo>
-		<CirclesWeb General="https://crosstalk.im/circles/$CIRCLECID$/settings/" General-SiteId="73625" Management="https://crosstalk.im/circles/$CIRCLECID$/settings/" Management-SiteId="73625" Membership="https://crosstalk.im/circles/$CIRCLECID$/settings/" Membership-SiteId="73625" Leave="https://crosstalk.im/circles/$CIRCLECID$/settings/" Leave-SiteId="73625" Disc="https://discord.gg/dumJwXTPxX" />
+		<CirclesWeb General="https://{{ settings.WEBSITE_HOST }}/circles/$CIRCLECID$/settings/" General-SiteId="73625" Management="https://{{ settings.WEBSITE_HOST }}/circles/$CIRCLECID$/settings/" Management-SiteId="73625" Membership="https://{{ settings.WEBSITE_HOST }}/circles/$CIRCLECID$/settings/" Membership-SiteId="73625" Leave="https://{{ settings.WEBSITE_HOST }}/circles/$CIRCLECID$/settings/" Leave-SiteId="73625" Disc="https://{{ settings.WEBSITE_HOST }}" />
 		<AnonymousIMSettings>
 			<SettingsPage URL="http://settings.messenger.live.com/applications/WebSettings.aspx" SiteID="76215" />
 		</AnonymousIMSettings>
@@ -494,32 +494,32 @@
 	</Simple>
 	<AbchCfg>
 		<abchconfig>
-			<url>https://ctsvcs.addressbook.ugnet.gay/abservice.asmx</url>
+			<url>https://{{ settings.ADDRESSBOOK_HOST }}/abservice.asmx</url>
 		</abchconfig>
 	</AbchCfg>
 	<AdMainConfig>
 		<AdDownloadTimeInMinutes>120</AdDownloadTimeInMinutes>
-		<AdMainCfg>http://ctsvcs.advertising.ugnet.gay/ads/banner</AdMainCfg>
+		<AdMainCfg>http://{{ settings.AD_HOST }}/ads/banner</AdMainCfg>
 	</AdMainConfig>
-	<AdPhoneConfig>http://ctsvcs.advertising.ugnet.gay/ads/banner</AdPhoneConfig>
+	<AdPhoneConfig>http://{{ settings.AD_HOST }}/ads/banner</AdPhoneConfig>
 	<LocalizedConfig Market="en-US">
 		<AdMainConfig>
-			<AdBanner20URL Refresh="390">http://ctsvcs.advertising.ugnet.gay/ads/banner</AdBanner20URL>
-			<AdBanner300URL Refresh="390" FooterPane="AlwaysShown">http://ctsvcs.advertising.ugnet.gay/ads/banner-lg</AdBanner300URL>
+			<AdBanner20URL Refresh="390">http://{{ settings.AD_HOST }}/ads/banner</AdBanner20URL>
+			<AdBanner300URL Refresh="390" FooterPane="AlwaysShown">http://{{ settings.AD_HOST }}/ads/banner-lg</AdBanner300URL>
 			<ConvoWinAd AdVisible="420" FooterPane="CollapsibleAndHideable" />
-			<ConvoWinAdBannerURL>http://ctsvcs.advertising.ugnet.gay/ads/banner</ConvoWinAdBannerURL>
+			<ConvoWinAdBannerURL>http://{{ settings.AD_HOST }}/ads/banner</ConvoWinAdBannerURL>
 			<StdAd AdVisible="60" VideoVisible="120" VideoDldIntvl="0">
-				<VideoFeeds FeedsListURL="http://ctsvcs.advertising.ugnet.gay/svcs/msn-video-feeds/"/>
+				<VideoFeeds FeedsListURL="http://{{ settings.AD_HOST }}/svcs/msn-video-feeds/"/>
 			</StdAd>
 			<TextAdRotation>420</TextAdRotation>
 			<TextAdRefresh>1</TextAdRefresh>
-			<TextAdServer>https://ctsvcs.advertising.ugnet.gay/ads/txt</TextAdServer>
+			<TextAdServer>https://{{ settings.AD_HOST }}/ads/txt</TextAdServer>
 			<Window Size="680" />
 		</AdMainConfig>
 		<AppDirConfig>
-			<AppDirPageURL>https://mactivities.msgrsvcs.ctsrv.gay/AppDirectory/Directory.aspx?L=en-us</AppDirPageURL>
-			<AppDirSeviceURL>https://mactivities.msgrsvcs.ctsrv.gay/AppDirectory/AppDirectory.asmx</AppDirSeviceURL>
-			<AppDirVersionURL>https://mactivities.msgrsvcs.ctsrv.gay/AppDirectory/GetAppdirVersion.aspx</AppDirVersionURL>
+			<AppDirPageURL>https://{{ settings.APPDIR_HOST }}/AppDirectory/Directory.aspx?L=en-us</AppDirPageURL>
+			<AppDirSeviceURL>https://{{ settings.APPDIR_HOST }}/AppDirectory/AppDirectory.asmx</AppDirSeviceURL>
+			<AppDirVersionURL>https://{{ settings.APPDIR_HOST }}/AppDirectory/GetAppdirVersion.aspx</AppDirVersionURL>
 		</AppDirConfig>
 		<AVPrerollAd FetchTimeout="6900" IntroLength="3000" MinAdLength="7000" URL="http://rad.msn.com/ADSAdClient31.dll?GetAd=&amp;PG=IMUSV1" />
 		<AVPrerollAd2 FetchTimeout="6900" IntroLength="0" MinAdLength="7000" URL="http://rad.msn.com/ADSAdClient31.dll?GetAd=&amp;PG=IMUVD1" />
@@ -615,22 +615,22 @@
 		</Contacts>
 		<ContactCard>
 			<AddToMySpaceSiteId>73625</AddToMySpaceSiteId>
-			<AddToMySpaceUrl>http://social.ugnet.gay/?addblogentry=true&amp;mkt=en-us</AddToMySpaceUrl>
+			<AddToMySpaceUrl>http://{{ settings.SOCIAL_HOST }}/?addblogentry=true&amp;mkt=en-us</AddToMySpaceUrl>
 			<ContactCardDisabled>false</ContactCardDisabled>
 			<CreateSpaceSiteId>73625</CreateSpaceSiteId>
-			<CreateSpaceUrl>http://social.ugnet.gay</CreateSpaceUrl>
+			<CreateSpaceUrl>http://{{ settings.SOCIAL_HOST }}</CreateSpaceUrl>
 			<GetItemVerUrl>http://storage.msn.com/storageservice/schematizedstore.asmx</GetItemVerUrl>
-			<GetXmlFeedUrl>http://cc.services.social.ugnet.gay/contactcard/contactcardservice.asmx</GetXmlFeedUrl>
-			<GetXmlFeedRpsUrl>http://cc.services.social.ugnet.gay/contactcard/contactcardservice.asmx</GetXmlFeedRpsUrl>
-			<GetXmlFeedRpsCidUrl>http://cid-%1!16.16I64X!.cc.services.social.ugnet.gay/contactcard/contactcardservice.asmx</GetXmlFeedRpsCidUrl>
+			<GetXmlFeedUrl>http://cc.services.{{ settings.SOCIAL_HOST }}/contactcard/contactcardservice.asmx</GetXmlFeedUrl>
+			<GetXmlFeedRpsUrl>http://cc.services.{{ settings.SOCIAL_HOST }}/contactcard/contactcardservice.asmx</GetXmlFeedRpsUrl>
+			<GetXmlFeedRpsCidUrl>http://cid-%1!16.16I64X!.cc.services.{{ settings.SOCIAL_HOST }}/contactcard/contactcardservice.asmx</GetXmlFeedRpsCidUrl>
 			<MySpaceSiteId>73625</MySpaceSiteId>
-			<MySpaceUrl>http://social.ugnet.gay</MySpaceUrl>
+			<MySpaceUrl>http://{{ settings.SOCIAL_HOST }}</MySpaceUrl>
 			<SpaceBaseSiteId>73625</SpaceBaseSiteId>
-			<SpaceBaseUrl>http://social.ugnet.gay</SpaceBaseUrl>
+			<SpaceBaseUrl>http://{{ settings.SOCIAL_HOST }}</SpaceBaseUrl>
 			<SpaceIntegrationEnabled>true</SpaceIntegrationEnabled>
 		</ContactCard>
 		<EditorialConfig>http://www.msn.com/infopane/messenger.armx</EditorialConfig>
-		<FlashUpgradeURL>http://storage.ugnet.gay/crosstalk-dist/client/all/tools/flashplayer_10_3r183_48_winax.exe</FlashUpgradeURL>
+		<FlashUpgradeURL>http://{{ settings.STORAGE_HOST }}/crosstalk-dist/client/all/tools/flashplayer_10_3r183_48_winax.exe</FlashUpgradeURL>
 		<FSS>
 			<SiteID>71659</SiteID>
 			<AddBuddyURL>https://fss.live.com/krl/AddContact.aspx</AddBuddyURL>
@@ -664,7 +664,7 @@
 			<General URL="http://www.microsoft.com/windowslivedevices/marketplace/default.aspx" />
 			<Webcam URL="http://www.microsoft.com/windowslivedevices/marketplace/default.aspx" />
 		</Marketplace>
-		<MessengerBlogURL>https://crosstalk.im/blog</MessengerBlogURL>
+		<MessengerBlogURL>https://{{ settings.WEBSITE_HOST }}/blog</MessengerBlogURL>
 		<MobileMessaging Mode="RPPNew" CarrierList="" CrossSell="1" CharMax="140" MainWindowButton="true" OperatorPage="http://get.live.com/messenger/mobile" ValidRoutes="US;CA" />
 		<MSNSearch>
 			<DesktopInstallURL>https://searx.ugnet.gay/searxng/search?q=$QUERY$&amp;ia=web</DesktopInstallURL>
@@ -680,16 +680,16 @@
 		</MSNSearch>
 		<MsnTodayConfig>
 			<MsnTodaySiteID>6528</MsnTodaySiteID>
-			<MsnTodayURL>https://today.msgrsvcs.ctsrv.gay/start?windowslive=1</MsnTodayURL>
+			<MsnTodayURL>https://{{ settings.TODAY_HOST }}/start?windowslive=1</MsnTodayURL>
 			<Flags>2</Flags>
 		</MsnTodayConfig>
 		<MusicIntegration URL="https://www.last.fm/search/tracks?q=$ARTIST$+$TITLE$"/>
 		<PartnerFeeds>
-			<Feed Name="MSN" URL="https://today.msgrsvcs.ctsrv.gay/start?windowslive=1" Owner="MSN" />
+			<Feed Name="MSN" URL="https://{{ settings.TODAY_HOST }}/start?windowslive=1" Owner="MSN" />
 			<Feed Name="GAMES" URL="http://windowsphone.xbox.com/en-US/MessengerGames/TabConfig" Owner="GAMES" />
 		</PartnerFeeds>
 		<PartnerFeedsSSL>
-			<Feed Name="MSN" URL="https://today.msgrsvcs.ctsrv.gay/start?windowslive=1" Owner="MSN" />
+			<Feed Name="MSN" URL="https://{{ settings.TODAY_HOST }}/start?windowslive=1" Owner="MSN" />
 			<Feed Name="GAMES" URL="http://windowsphone.xbox.com/en-US/MessengerGames/TabConfig" Owner="GAMES" />
 		</PartnerFeedsSSL>
 		<PE Tracking="http://trackingw4.technodesignip.com/?ContentID=$ContentID$&amp;Click=$Click$" />
@@ -702,7 +702,7 @@
 			<ReportAbuseSID />
 			<ReportAbuseURL>http://support.live.com/default.aspx?productkey=wlmessengerabuse</ReportAbuseURL>
 			<BiciReportAbuseApiUrl>https://home.live.com/handlers/ReportAbuse.mvc</BiciReportAbuseApiUrl>
-			<BiciReportAbuseApiServiceTarget>social.ugnet.gay</BiciReportAbuseApiServiceTarget>
+			<BiciReportAbuseApiServiceTarget>{{ settings.SOCIAL_HOST }}</BiciReportAbuseApiServiceTarget>
 		</ReportAbuse>
 		<RL>
 			<ViewProfileURL>http://g.msn.com/5meen_us/106?%1&amp;Plcid=%2!hs!&amp;%3&amp;Country=%4!hs!&amp;BrandID=%5&amp;passport=%6</ViewProfileURL>
@@ -750,14 +750,14 @@
 			<slots>
 				{% for tab in tabs %}
 					<URL id="{{ tab.id }}">
-						https://ctsvcs.advertising.ugnet.gay/svcs/tab={{ tab.id }}
+						https://{{ settings.AD_HOST }}/svcs/tab={{ tab.id }}
 					</URL>
 				{% endfor %}
 				</slots>
   		</TabConfig>
 		<TermsOfUse>
 			<TermsOfUseSID>956</TermsOfUseSID>
-			<TermsOfUseURL>https://crosstalk.im/tos</TermsOfUseURL>
+			<TermsOfUseURL>https://{{ settings.WEBSITE_HOST }}/tos</TermsOfUseURL>
 		</TermsOfUse>
 		<UPUX>
 			<Product id="8" PartnerID="TD" ProductName="Winks" BrandName="Quebles Winks">

front/msn/http/tmpl/config/MsgrConfig.wlm.8.xml

@@ -73,51 +73,51 @@
 	</TabConfig>
 	<AbchCfg>
 		<abchconfig>
-			<url>https://ctsvcs.addressbook.ugnet.gay/abservice.asmx</url>
+			<url>https://{{ settings.ADDRESSBOOK_HOST }}/abservice.asmx</url>
 		</abchconfig>
 	</AbchCfg>
 	<LocalizedConfig Market="en-US">
 		<AdMainConfig>
 			<TextAdRefresh>1</TextAdRefresh>
-			<TextAdServer>http://ctsvcs.advertising.ugnet.gay/ads/txt</TextAdServer>
-			<AdBanner20URL Refresh="300">http://ctsvcs.advertising.ugnet.gay/ads/banner</AdBanner20URL>
+			<TextAdServer>http://{{ settings.AD_HOST }}/ads/txt</TextAdServer>
+			<AdBanner20URL Refresh="300">http://{{ settings.AD_HOST }}/ads/banner</AdBanner20URL>
 			<StdAd AdVisible="60" VideoVisible="120" VideoDldIntvl="0">
-				<VideoFeeds FeedsListURL="http://ctsvcs.advertising.ugnet.gay/svcs/msn-video-feeds/"/>
+				<VideoFeeds FeedsListURL="http://{{ settings.AD_HOST }}/svcs/msn-video-feeds/"/>
 			</StdAd>
 		</AdMainConfig>
 		<AppDirConfig>
-			<AppDirPageURL>http://mactivities.msgrsvcs.ctsrv.gay/AppDirectory/Directory.aspx?L=en-us</AppDirPageURL>
-			<AppDirSeviceURL>http://mactivities.msgrsvcs.ctsrv.gay/AppDirectory/AppDirectory.asmx</AppDirSeviceURL>
-			<AppDirVersionURL>http://mactivities.msgrsvcs.ctsrv.gay/AppDirectory/GetAppdirVersion.aspx</AppDirVersionURL>
+			<AppDirPageURL>http://{{ settings.APPDIR_HOST }}/AppDirectory/Directory.aspx?L=en-us</AppDirPageURL>
+			<AppDirSeviceURL>http://{{ settings.APPDIR_HOST }}/AppDirectory/AppDirectory.asmx</AppDirSeviceURL>
+			<AppDirVersionURL>http://{{ settings.APPDIR_HOST }}/AppDirectory/GetAppdirVersion.aspx</AppDirVersionURL>
 		</AppDirConfig>
 		<Contacts>
 			<ABPendingRequests>
-				<url>http://social.ugnet.gay/PendingRequests.aspx?mkt=en-us&amp;partner=Messenger</url>
+				<url>http://{{ settings.SOCIAL_HOST }}/PendingRequests.aspx?mkt=en-us&amp;partner=Messenger</url>
 				<siteid>73625</siteid>
 				<height>672</height>
 				<width>808</width>
 			</ABPendingRequests>
 			<BuddysSocialNetwork>
-				<url>http://social.ugnet.gay/NetworkExplorer.aspx?mkt=en-us&amp;partner=Messenger&amp;cid=%1</url>
+				<url>http://{{ settings.SOCIAL_HOST }}/NetworkExplorer.aspx?mkt=en-us&amp;partner=Messenger&amp;cid=%1</url>
 				<siteid>73625</siteid>
 				<height>674</height>
 				<width>808</width>
 			</BuddysSocialNetwork>
 			<EditMyLiveContactProfile>
-				<url>http://social.ugnet.gay/profile.aspx?action=edit&amp;mode=activecontacts&amp;mkt=en-us&amp;partner=Messenger</url>
+				<url>http://{{ settings.SOCIAL_HOST }}/profile.aspx?action=edit&amp;mode=activecontacts&amp;mkt=en-us&amp;partner=Messenger</url>
 				<siteid>73625</siteid>
 			</EditMyLiveContactProfile>
 			<EditMyProfile>
-				<url>http://social.ugnet.gay/profile.aspx?action=edit&amp;partner=Messenger&amp;mkt=en-us</url>
+				<url>http://{{ settings.SOCIAL_HOST }}/profile.aspx?action=edit&amp;partner=Messenger&amp;mkt=en-us</url>
 				<siteid>73625</siteid>
 			</EditMyProfile>
 			<Profile>
-				<url>http://social.ugnet.gay/Profile.aspx?partner=Messenger&amp;mkt=en-us&amp;mem=%ls</url>
-				<urlcid>http://social.ugnet.gay/Profile.aspx?partner=Messenger&amp;mkt=en-us&amp;cid=%ls</urlcid>
+				<url>http://{{ settings.SOCIAL_HOST }}/Profile.aspx?partner=Messenger&amp;mkt=en-us&amp;mem=%ls</url>
+				<urlcid>http://{{ settings.SOCIAL_HOST }}/Profile.aspx?partner=Messenger&amp;mkt=en-us&amp;cid=%ls</urlcid>
 				<siteid>73625</siteid>
 			</Profile>
 			<SocialNetwork>
-				<url>http://social.ugnet.gay/NetworkSetup.aspx?mkt=en-us&amp;partner=Messenger</url>
+				<url>http://{{ settings.SOCIAL_HOST }}/NetworkSetup.aspx?mkt=en-us&amp;partner=Messenger</url>
 				<siteid>73625</siteid>
 				<height>674</height>
 				<width>808</width>
@@ -126,21 +126,21 @@
 		</Contacts>
 		<ContactCard>
 			<AddToMySpaceSiteId>45930</AddToMySpaceSiteId>
-			<AddToMySpaceUrl>http://social.ugnet.gay/?addblogentry=true&amp;mkt=nl-nl</AddToMySpaceUrl>
+			<AddToMySpaceUrl>http://{{ settings.SOCIAL_HOST }}/?addblogentry=true&amp;mkt=nl-nl</AddToMySpaceUrl>
 			<ContactCardDisabled>false</ContactCardDisabled>
 			<CreateSpaceSiteId>45930</CreateSpaceSiteId>
-			<CreateSpaceUrl>http://social.ugnet.gay/signup.aspx?mkt=nl-nl</CreateSpaceUrl>
+			<CreateSpaceUrl>http://{{ settings.SOCIAL_HOST }}/signup.aspx?mkt=nl-nl</CreateSpaceUrl>
 			<GetItemVerUrl>http://storage.msn.com/storageservice/schematizedstore.asmx</GetItemVerUrl>
-			<GetXmlFeedUrl>http://services.social.ugnet.gay/contactcard/contactcardservice.asmx</GetXmlFeedUrl>
+			<GetXmlFeedUrl>http://services.{{ settings.SOCIAL_HOST }}/contactcard/contactcardservice.asmx</GetXmlFeedUrl>
 			<MySpaceSiteId>45930</MySpaceSiteId>
-			<MySpaceUrl>http://social.ugnet.gay/homepage.aspx</MySpaceUrl>
+			<MySpaceUrl>http://{{ settings.SOCIAL_HOST }}/homepage.aspx</MySpaceUrl>
 			<SpaceBaseSiteId>45930</SpaceBaseSiteId>
-			<SpaceBaseUrl>http://social.ugnet.gay</SpaceBaseUrl>
+			<SpaceBaseUrl>http://{{ settings.SOCIAL_HOST }}</SpaceBaseUrl>
 			<SpaceIntegrationEnabled>true</SpaceIntegrationEnabled>
 		</ContactCard>
-		<FlashUpgradeURL>http://storage.ugnet.gay/crosstalk-dist/client/all/tools/flashplayer_10_3r183_48_winax.exe</FlashUpgradeURL>
+		<FlashUpgradeURL>http://{{ settings.STORAGE_HOST }}/crosstalk-dist/client/all/tools/flashplayer_10_3r183_48_winax.exe</FlashUpgradeURL>
 		<GLinkMarket v="en_us"/>
-		<MessengerBlogURL>https://crosstalk.im/blog</MessengerBlogURL>
+		<MessengerBlogURL>https://{{ settings.WEBSITE_HOST }}/blog</MessengerBlogURL>
 		<MSNSearch>
 			<ImagesURL>https://searx.ugnet.gay/searxng/search?q=$QUERY$&amp;ia=images</ImagesURL>
 			<NewsURL>https://searx.ugnet.gay/searxng/search?q=$QUERY$+news&amp;ia=news</NewsURL>
@@ -150,7 +150,7 @@
 		</MSNSearch>
 		<MsnTodayConfig>
 			<MsnTodaySiteID>6528</MsnTodaySiteID>
-			<MsnTodayURL>http://today.msgrsvcs.ctsrv.gay/start?wlm=1</MsnTodayURL>
+			<MsnTodayURL>http://{{ settings.TODAY_HOST }}/start?wlm=1</MsnTodayURL>
 			<Flags>2</Flags>
 		</MsnTodayConfig>
 		<MusicIntegration URL="https://www.last.fm/search/tracks?q=$ARTIST$+$TITLE$"/>
@@ -158,15 +158,15 @@
 		<PSTNOut Hidden="1"/>
 		<ReportAbuse>
 			<ReportAbuseSID/>
-			<ReportAbuseURL>https://crosstalk.im/report</ReportAbuseURL>
+			<ReportAbuseURL>https://{{ settings.WEBSITE_HOST }}/report</ReportAbuseURL>
 		</ReportAbuse>
 		<RL>
-			<ViewProfileURL>https://crosstalk.im/account/settings</ViewProfileURL>
+			<ViewProfileURL>https://{{ settings.WEBSITE_HOST }}/account/settings</ViewProfileURL>
 		</RL>
 		<SDrive Enabled="true"/>
 		<TermsOfUse>
 			<TermsOfUseSID>956</TermsOfUseSID>
-			<TermsOfUseURL>https://crosstalk.im/tos</TermsOfUseURL>
+			<TermsOfUseURL>https://{{ settings.WEBSITE_HOST }}/tos</TermsOfUseURL>
 		</TermsOfUse>
 		<VoiceClip Hidden="0"/>
 		<VoiceVideo>

front/msn/http/tmpl/oim/Fault.validation.xml

@@ -4,7 +4,7 @@
 		<soap:Fault>
 			<faultcode>soap:Client</faultcode>
 			<faultstring>Schema validation error</faultstring>
-			<faultactor>https://cts.storage.ugnet.gay/rsi/rsi.asmx</faultactor>
+			<faultactor>https://{{ settings.USERSTORAGE_HOST }}/rsi/rsi.asmx</faultactor>
 			<detail />
 		</soap:Fault>
 	</soap:Body>

front/msn/http/tmpl/storageservice/GetProfileResponse.xml

@@ -34,7 +34,7 @@
 									<DocumentStreamName>UserTileStatic</DocumentStreamName>
 									<MimeType>image/{{ mime }}</MimeType>
 									<DataSize>{{ size_static }}</DataSize>
-									<PreAuthURL>https://{{ host }}/avatar/{{ user.uuid }}/static</PreAuthURL>
+									<PreAuthURL>https://{{ host }}/avatar/{{ user.uuid }}/msn</PreAuthURL>
 									<WriteMode>Overwrite</WriteMode>
 									<StreamDataStatus>None</StreamDataStatus>
 									<StreamStatus>None</StreamStatus>
@@ -68,7 +68,7 @@
 						<DisplayName>{{ roaming_info.name }}</DisplayName>
 						<DisplayNameLastModified>{{ date_format(roaming_info.name_last_modified) }}</DisplayNameLastModified>
 					{% endif %}
-					<StaticUserTilePublicURL>https://{{ host }}/avatar/{{ user.uuid }}/static</StaticUserTilePublicURL>
+					<StaticUserTilePublicURL>https://{{ host }}/avatar/{{ user.uuid }}/msn</StaticUserTilePublicURL>
 				</ExpressionProfile>
 			</GetProfileResult>
 		</GetProfileResponse>

front/msn/http/tmpl/svcs/MSNToday.html

@@ -100,7 +100,7 @@
 		</div>
 		<div class="content">
 			<h3>Welcome! Thank you for using CrossTalk!</h3>
-			<p>If you find any issues, have suggestions for any new features, or just want to hang out with fellow CrossTalk users, check out our <a href="https://discord.gg/dumJwXTPxX">Discord server</a>.</p>
+			<p>If you find any issues, have suggestions for any new features, or just want to hang out with fellow CrossTalk users, check out our community <a href="https://crosstalk.im/community">IRC/Forums</a>.</p>
 			<p>This page will soon be redesigned. Stay tuned!</p>
 		</div>
 	</body>

front/msn/http/tmpl/svcs/videofeedroot.xml

@@ -2,7 +2,7 @@
 <rss version="2.0">
 	<channel>
 		<title>CrossTalk Video</title>
-		<link>https://crosstalk.im</link>
+		<link>https://{{ settings.WEBSITE_HOST }}</link>
 		<description>CrossTalk Video</description>
 		<language>en-us</language>
 		<copyright>2023 - 2026 the undergr0und</copyright>
@@ -11,7 +11,7 @@
 		<ttl>0</ttl>
 		<item>
 			<title>Testing</title>
-			<link>http://ctsvcs.advertising.ugnet.gay/svcs/feed</link>
+			<link>http://{{ settings.AD_HOST }}/svcs/feed</link>
 			<description>Test feed</description>
 			<pubDate>Thu, 05 Sep 2024 00:00:00 GMT</pubDate>
 			<category>MSN_Microsoft</category>

front/msn/http/tmpl/whatsnew/WhatsNewResponse.xml

@@ -54,7 +54,7 @@
 						<CanMergeItemsWithSameObjectID>false</CanMergeItemsWithSameObjectID>
 					</RecentActivityTemplateContainer>
 				</Templates>
-				<FeedUrl>http://crosstalk.im</FeedUrl>
+				<FeedUrl>http://{{ settings.WEBSITE_HOST }}</FeedUrl>
 			</GetContactsRecentActivityResult>
 		</GetContactsRecentActivityResponse>
 	</soap:Body>

front/msn/misc.py

@@ -44,6 +44,9 @@ def build_presence_notif(
 	
 	ctc_sess = first_in_iterable(backend.util_get_sessions_by_user(head))
 	
+	head_display_email = backend.util_get_display_email(head)
+	user_me_display_email = backend.util_get_display_email(user_me)
+	
 	# TODO: This is insanely ugly.
 	if dialect >= 19:
 		cm = None # type: Optional[str]
@@ -61,7 +64,7 @@ def build_presence_notif(
 			cm = NFY_PUT_PRESENCE_USER_S_CM.format(cm = encode_xml_he(status.media or '', dialect))
 			nfy_rst += NFY_PUT_PRESENCE_USER_S_PE.format(
 				msnobj = encode_xml_he(ctc_sess.front_data.get('msn_msnobj') or '', dialect),
-				name = status.name or head.email, message = status.message,
+				name = status.name or head_display_email, message = status.message,
 				ddp = encode_xml_he(ctc_sess.front_data.get('msn_msnobj_ddp') or '', dialect),
 				colorscheme = encode_xml_he(ctc_sess.front_data.get('msn_colorscheme') or '', dialect),
 				scene = encode_xml_he(ctc_sess.front_data.get('msn_msnobj_scene') or '', dialect),
@@ -133,7 +136,7 @@ def build_presence_notif(
 		)
 		
 		nfy_payload = encode_payload(NFY_PUT_PRESENCE,
-			to = user_me.email, from_email = head.email, cl = len(nfy_presence_body), payload = nfy_presence_body,
+			to = user_me_display_email, from_email = head_display_email, cl = len(nfy_presence_body), payload = nfy_presence_body,
 		)
 		
 		yield ('NFY', 'PUT', nfy_payload)
@@ -142,9 +145,9 @@ def build_presence_notif(
 	if status.substatus is not old_substatus:
 		if (is_offlineish or (circle is not None and circle.memberships[head.uuid].blocking)) and head is not user_me:
 			if dialect >= 18:
-				reply = ('FLN', encode_email_networkid(head.email, None, circle = circle)) # type: Tuple[Any, ...]
+				reply = ('FLN', encode_email_networkid(head_display_email, None, circle = circle)) # type: Tuple[Any, ...]
 			else:
-				reply = ('FLN', head.email, (int(NetworkID.WINDOWS_LIVE) if 14 <= dialect <= 17 else None))
+				reply = ('FLN', head_display_email, (int(NetworkID.WINDOWS_LIVE) if 14 <= dialect <= 17 else None))
 			
 			if 14 <= dialect <= 15:
 				reply += ('0',)
@@ -175,9 +178,9 @@ def build_presence_notif(
 			rst.append(MSNObj(ctc_sess.front_data.get('msn_msnobj') or '<msnobj/>'))
 		
 		if dialect >= 18:
-			yield (*frst, msn_status.name, encode_email_networkid(head.email, None, circle = circle), status.name, *rst)
+			yield (*frst, msn_status.name, encode_email_networkid(head_display_email, None, circle = circle), status.name, *rst)
 		else:
-			yield (*frst, msn_status.name, head.email, (int(NetworkID.WINDOWS_LIVE) if 14 <= dialect <= 17 else None), status.name, *rst)
+			yield (*frst, msn_status.name, head_display_email, (int(NetworkID.WINDOWS_LIVE) if 14 <= dialect <= 17 else None), status.name, *rst)
 	
 	# MSNP16/18 requires `UBX`s when a user changes any sort of state (this makes MPoP and Circles work effectively)
 	if dialect < 11 or (not update_info and dialect < 16):
@@ -190,9 +193,9 @@ def build_presence_notif(
 	).encode('utf-8')
 	
 	if dialect >= 18:
-		yield ('UBX', encode_email_networkid(head.email, None, circle = circle), ubx_payload)
+		yield ('UBX', encode_email_networkid(head_display_email, None, circle = circle), ubx_payload)
 	else:
-		yield ('UBX', head.email, (int(NetworkID.WINDOWS_LIVE) if 14 <= dialect <= 17 else None), ubx_payload)
+		yield ('UBX', head_display_email, (int(NetworkID.WINDOWS_LIVE) if 14 <= dialect <= 17 else None), ubx_payload)
 
 def encode_email_networkid(email: str, networkid: Optional[NetworkID], *, circle: Optional['Circle'] = None) -> str:
 	result = '{}:{}'.format(int(networkid or NetworkID.WINDOWS_LIVE), email)
@@ -350,54 +353,46 @@ def encode_payload(tmpl: str, **kwargs: Any) -> bytes:
 	return tmpl.format(**kwargs).replace('\n', '\r\n').encode('utf-8')
 
 def gen_chal_response(chal: str, id: str, id_key: str, *, msnp11: bool = False) -> str:
-	key_hash = md5((chal + id_key).encode())
-	
+	key_digest = md5((chal + id_key).encode('ascii')).digest()
+
 	if not msnp11:
-		return key_hash.hexdigest()
-	
-	key_digest = key_hash.digest()
-	key_array = []
-	for i in range(0, len(key_digest), 4):
-		key_array.append(struct.unpack('<I', key_digest[i:i+4])[0] & 0x7FFFFFFF)
-	chl = chal.encode() + id.encode()
-	chl += (b'0' * (4 - (len(chl) % 4)))
-	chl_array = []
-	for j in range(0, len(chl), 4):
-		chl_array.append(struct.unpack('<I', chl[j:j+4])[0])
-	
+		return key_digest.hex()
+
+	private_seed = [
+		struct.unpack('<I', key_digest[i:i+4])[0] & 0x7FFFFFFF
+		for i in range(0, 16, 4)
+	]
+
+	pub_bytes = (chal + id).encode('ascii')
+	pub_bytes += b'0' * (8 - (len(pub_bytes) % 8))
+	public_seed = [
+		struct.unpack('<I', pub_bytes[i:i+4])[0] & 0x7FFFFFFF
+		for i in range(0, len(pub_bytes), 4)
+	]
+
 	high = 0
 	low = 0
-	
-	for n in range(0, len(chl_array) - 1, 2):
-		temp = chl_array[n]
-		temp = (0x0E79A9C1 * temp) % 0x7FFFFFFF
-		temp += high
-		temp = key_array[0] * temp + key_array[1]
-		temp = temp % 0x7FFFFFFF
-		
-		high = chl_array[n + 1]
-		high = (high + temp) % 0x7FFFFFFF
-		high = key_array[2] * high + key_array[3]
-		high = high % 0x7FFFFFFF
-		
+	for i in range(0, len(public_seed), 2):
+		temp = (public_seed[i] * 0x0E79A9C1) % 0x7FFFFFFF
+		temp = (temp + high)
+		temp = (temp * private_seed[0] + private_seed[1]) % 0x7FFFFFFF
+
+		high = (public_seed[i + 1] + temp) % 0x7FFFFFFF
+		high = (high * private_seed[2] + private_seed[3]) % 0x7FFFFFFF
+
 		low += high + temp
-	
-	high = (high + key_array[1]) % 0x7FFFFFFF
-	low = (low + key_array[3]) % 0x7FFFFFFF
-	
-	key_array[0] ^= high
-	key_array[1] ^= low
-	key_array[2] ^= high
-	key_array[3] ^= low
-	
-	final = (
-		binascii.hexlify(struct.pack('<I', key_array[0]))
-		+ binascii.hexlify(struct.pack('<I', key_array[1]))
-		+ binascii.hexlify(struct.pack('<I', key_array[2]))
-		+ binascii.hexlify(struct.pack('<I', key_array[3]))
-	).decode('utf-8')
-	
-	return final
+
+	high = struct.unpack('<I', struct.pack('>I', (high + private_seed[1]) % 0x7FFFFFFF))[0]
+	low = struct.unpack('<I', struct.pack('>I', (low + private_seed[3]) % 0x7FFFFFFF))[0]
+	key = struct.unpack('<Q', struct.pack('>Q', (high << 32) | (low & 0xFFFFFFFF)))[0]
+
+	result_high = struct.unpack('<Q', key_digest[0:8])[0] ^ key
+	result_low = struct.unpack('<Q', key_digest[8:16])[0] ^ key
+
+	return (
+		struct.pack('<Q', result_high).hex() +
+		struct.pack('<Q', result_low).hex()
+	)
 
 def generate_rps_key(key: bytes, msg: bytes) -> bytes:
 	hash1 = hmac.new(key, msg, sha1).digest()
@@ -686,6 +681,7 @@ class Err:
 	PrincipalNotOnContactList = 216
 	PrincipalNotOnline = 217
 	AlreadyInMode = 218
+	MutuallyExclusiveList = 219
 	GroupInvalid = 224
 	PrincipalNotInGroup = 225
 	GroupAlreadyExists = 228
@@ -707,7 +703,7 @@ class Err:
 	InvalidDomain = 917
 	ChildAccountNotAuthorized = 923
 	NeedsHostUpdate = 929
-	AccountNotVerified = 924
+	AccountNotVerified = 926
 	InvalidCircleMembership = 933
 	
 	@classmethod

front/msn/msnp_ns.py

@@ -34,13 +34,14 @@ MSNP_DIALECTS = ['MSNP{}'.format(d) for d in (
 
 class MSNPCtrlNS(MSNPCtrl):
 	__slots__ = (
-		'backend', 'dialect', 'usr_email', 'bs', 'client', 'syn_ser', 'gcf_sent', 'syn_sent', 'iln_sent', 'is_third_party_client',
-		'challenge', 'rps_challenge', 'circle_authenticated', 'new_circles', 'initial_adl_sent', 'circle_adl_sent', 'password', 'cs', 'is_on_old_reroute', 'email_conflict'
+		'backend', 'dialect', 'usr_email', 'email_for_validation', 'bs', 'client', 'syn_ser', 'gcf_sent', 'syn_sent', 'iln_sent', 'is_third_party_client',
+		'challenge', 'rps_challenge', 'circle_authenticated', 'new_circles', 'initial_adl_sent', 'circle_adl_sent', 'password', 'cs', 'email_conflict'
 	)
 
 	backend: Backend
 	dialect: int
-	usr_email: Optional[str]
+	usr_email: Optional[str] # user's real e-mail
+	email_for_validation: Optional[str] # for validation etc
 	bs: Optional[BackendSession]
 	client: Client
 	is_third_party_client: bool
@@ -56,7 +57,6 @@ class MSNPCtrlNS(MSNPCtrl):
 	circle_adl_sent: bool
 	password: Optional[str]
 	cs: Optional[ChatSession]
-	is_on_old_reroute: bool
 	email_conflict: bool
 
 	def __init__(self, logger: Logger, via: str, backend: Backend) -> None:
@@ -64,6 +64,7 @@ class MSNPCtrlNS(MSNPCtrl):
 		self.backend = backend
 		self.dialect = 0
 		self.usr_email = None
+		self.email_for_validation = None
 		self.bs = None
 		self.password = None
 		self.client = Client('msn', '?', via)
@@ -79,7 +80,6 @@ class MSNPCtrlNS(MSNPCtrl):
 		self.initial_adl_sent = False
 		self.circle_adl_sent = False
 		self.cs = None
-		self.is_on_old_reroute = False
 		self.email_conflict = False
 	
 	def _on_close(self) -> None:
@@ -158,6 +158,9 @@ class MSNPCtrlNS(MSNPCtrl):
 						return
 
 					uuid = backend.user_service.login(email, pwd)
+					if uuid is None and email.lower().endswith('@crosstalk.im'):
+						username = email.lower()[:-len('@crosstalk.im')]
+						uuid = backend.user_service.login_with_username(username, pwd)
 					if uuid is not None:
 						bs = backend.login(
 							uuid,
@@ -173,7 +176,7 @@ class MSNPCtrlNS(MSNPCtrl):
 				self.send_reply(Err.AuthFail, trid)
 				self.close()
 			else:
-				self._auth_final_step(trid, token or '', None)
+				self._auth_final_step(trid, email, token or '', None)
 		
 		if authtype == 'SHA':
 			if dialect < 18:
@@ -186,19 +189,13 @@ class MSNPCtrlNS(MSNPCtrl):
 			signedticket = args[0]
 			if stage == 'A':
 				#>>> USR trid SHA A b64_signedticket
-				# Commenting out temporarily due to a bug where a given account's ticket is always deemed invalid if the server restarts while it's connected
-				# TODO: Fix that
-				#if signedticket != base64.b64encode(gen_signedticket_xml(bs, backend).encode('utf-8')).decode('utf-8'):
-				#	self.circle_authenticated = False
-				#	self.send_reply(Err.AuthFail, trid)
-				#	return
 				self.circle_authenticated = True
-				self.send_reply('USR', trid, 'OK', self.usr_email, 0, 0)
+				self.send_reply('USR', trid, 'OK', self.backend.util_get_display_email(bs.user), 0, 0)
 				
 				if self.circle_authenticated:
 					for circle in self.new_circles:
 						self.send_reply('NFY', 'PUT', encode_payload(PAYLOAD_MSG_7,
-							email = _encode_email_epid(bs.user.email, bs.front_data.get('msn_pop_id')), chat_id = circle.chat_id,
+							email = _encode_email_epid(self.backend.util_get_display_email(bs.user), bs.front_data.get('msn_pop_id')), chat_id = circle.chat_id,
 						))
 					self.new_circles.clear()
 			return
@@ -218,14 +215,39 @@ class MSNPCtrlNS(MSNPCtrl):
 					self.send_reply(Err.AuthFail, trid)
 					self.close()
 					return
+
+				self.usr_email = email
+				
+				username, domain = email.split('@', 1)
+				
+				if dialect == 2:
+					uuid = backend.user_service.get_uuid_username(username)
+					if uuid is None:
+						uuid = backend.user_service.get_uuid(email)
+					if uuid is None:
+						self.send_reply(Err.AuthFail, trid)
+						self.close()
+						return
+					user = backend.user_service.get(uuid)
+					email = self.backend.util_get_display_email(user)
+				
 				salt = backend.user_service.msn_get_md5_salt(email)
+				if salt is None and email.lower().endswith('@crosstalk.im'):
+					real_uuid = backend.user_service.get_uuid_username(email.lower()[:-len('@crosstalk.im')])
+					if real_uuid is not None:
+						real_user = backend.user_service.get(real_uuid)
+						if real_user is not None:
+							salt = backend.user_service.msn_get_md5_salt(real_user.email)
+							email = real_user.email
+							self.email_for_validation = email
+				else:
+					self.email_for_validation = self.usr_email
 				if salt is None:
 					# Account is not enabled for login via MD5; send `USR S` with Unix time as salt simply to
 					# keep MSNP `USR` flow consistent (`USR I` doesn't validate existence of email, but rather
 					# whether its format it correct, `USR S` actually does account checks)
 					# TODO: Can we pass an informative message to user?
 					salt = str(time.time())
-				self.usr_email = email
 				self.send_reply('USR', trid, authtype, 'S', salt)
 				return
 			if stage == 'S':
@@ -237,13 +259,14 @@ class MSNPCtrlNS(MSNPCtrl):
 					return
 				
 				md5_hash = args[0]
-				usr_email = self.usr_email
+				usr_email = self.usr_email.lower()
 				assert usr_email is not None
-				uuid = backend.user_service.msn_login_md5(usr_email, md5_hash)
+				uuid = backend.user_service.msn_login_md5(self.email_for_validation, md5_hash)
 				if uuid is not None:
 					self.bs = backend.login(uuid, self.client, BackendEventHandler(self.backend.loop, self), option = LoginOption.BootOthers)
 					token, _ = backend.login_auth_service.create_token('ns/login', [uuid, None], lifetime = 86400)
-				self._auth_final_step(trid, token or '', None)
+				print(f"usr_email during MD5 stage S: {usr_email}")
+				self._auth_final_step(trid, usr_email, token or '', None)
 				return
 		
 		if authtype == 'TWN':
@@ -260,8 +283,6 @@ class MSNPCtrlNS(MSNPCtrl):
 					self.send_reply(Err.AuthFail, trid)
 					self.close()
 					return
-				#extra = ('ct={},rver=5.5.4177.0,wp=FS_40SEC_0_COMPACT,lc=1033,id=507,ru=http://messenger.msn.com,\
-				#tw=0,kpp=1,kv=4,ver=2.1.6000.1,rn=1lgjBfIL,tpf=b0735e3a873dfb5e75054465196398e0'.format(int(time())),)
 				if dialect >= 13:
 					self.send_reply('GCF', 0, SHIELDS_MSNP13)
 				self.send_reply('USR', trid, authtype, 'S', 'ct=1,rver=1,wp=FS_40SEC_0_COMPACT,lc=1,id=1')
@@ -284,7 +305,7 @@ class MSNPCtrlNS(MSNPCtrl):
 					uuid = tpl[0]
 					assert uuid is not None
 					self.bs = backend.login(uuid, self.client, BackendEventHandler(self.backend.loop, self), option = LoginOption.BootOthers)
-				self._auth_final_step(trid, token, None)
+				self._auth_final_step(trid, usr_email, token, None)
 				return
 		
 		if authtype == 'SSO':
@@ -301,15 +322,14 @@ class MSNPCtrlNS(MSNPCtrl):
 					self.send_reply(Err.AuthFail, trid)
 					self.close()
 					return
-				# https://web.archive.org/web/20100819015007/http://msnpiki.msnfanatic.com/index.php/MSNP15:SSO
+				# https://yellows111.darkok.xyz/msnp/authschemes/sso.html
 				self.rps_challenge = base64.b64encode(secrets.token_bytes(48))
 				
 				self.send_reply('GCF', 0, SHIELDS_MSNP13)
 				self.send_reply('USR', trid, authtype, 'S', 'MBI_KEY_OLD', self.rps_challenge.decode('utf-8'))
 				return
 			if stage == 'S':
-				#>>> USR trid SSO S auth_token [b64_response; not included when MSIDCRL-patched clients login]
-				#>>> USR trid SSO S auth_token b64_response machineguid (MSNP >= 16)
+				#>>> USR trid SSO S auth_token [b64_response] [machineguid]
 				if backend.maintenance_mode:
 					self.send_reply(Err.InternalServerError, trid)
 					self.close()
@@ -414,25 +434,42 @@ class MSNPCtrlNS(MSNPCtrl):
 					else:
 						option = LoginOption.BootOthers
 					self.bs = backend.login(uuid, self.client, BackendEventHandler(self.backend.loop, self), option = option)
-				self._auth_final_step(trid, token, machineguid)
+				self._auth_final_step(trid, usr_email, token, machineguid)
 				return
 	
-	def _auth_final_step(self, trid: str, token: str, machineguid: Optional[str]) -> None:
+	def _auth_final_step(self, trid: str, attempted_email: str, token: str, machineguid: Optional[str]) -> None:
 		from cryptography.hazmat.backends import default_backend
 		from cryptography.hazmat.primitives.asymmetric import rsa
 		
 		bs = self.bs
 		dialect = self.dialect
-		
-		if bs is None or bs.user.email.lower() != self.usr_email.lower():
+		print(f"bs: {bs}")
+		if bs is None:
 			self.send_reply(Err.AuthFail, trid)
 			self.close()
 			return
+
+		user = bs.user
+		email = self.backend.util_get_display_email(user)
+
+		usr_email_lower = (self.usr_email or '').lower()
+		alias_email = '{}@crosstalk.im'.format(user.username).lower()
+		print(f"email: {email}")
+		print(f"attempted email: {attempted_email}")
+		print(f"self.usr_email: {self.usr_email}")
+		if usr_email_lower not in (user.email.lower(), alias_email, email.lower()):
+			self.send_reply(Err.AuthFail, trid)
+			self.close()
+			return
+		else:
+			if user.alias_active and attempted_email != alias_email:  #or user.alias_active and email == user.email.lower():
+				self.send_reply(Err.AuthFail, trid)
+				self.close()
+				return
 		
 		self.backend.util_set_sess_token(bs, token)
 		
 		bs.front_data['msn'] = True
-		
 		bs.front_data['windowslive_chats'] = {}
 		
 		if dialect >= 16:
@@ -443,9 +480,6 @@ class MSNPCtrlNS(MSNPCtrl):
 			public_exponent = 65537, key_size = 2048, backend = default_backend(), # type: ignore
 		)
 
-		user = bs.user
-		email = user.email
-
 		now = datetime.now()
 
 		if dialect < 10:
@@ -453,27 +487,40 @@ class MSNPCtrlNS(MSNPCtrl):
 		else:
 			args = ()
 		if dialect >= 6:
-			verified = user.verified_to_login
+			verified = True # TODO: check for verified_to_login when email verification is finished
 			args += ((1 if verified else 0),)
 		if dialect >= 8:
 			args += (0,)
 
+		with open('config/restricted-emails.json', 'r') as file:
+			restricted_emails = json.load(file)
+			email_domain = user.email.lower().split('@')[-1]
+			if email_domain in restricted_emails or email_domain in disposable_emails:
+				self.send_reply(Err.InvalidDomain, trid)
+				self.send_reply('NOT', encode_payload(PAYLOAD_MSG_10,
+					email = email, icon = '', url = 'https://crosstalk.im/alias-prep',
+					msg = "Your e-mail address contains a disallowed domain and needs to be changed before you can log in. Click this alert to learn more.",
+				))
+				self.close()
+				return
+
 		if user.suspended: 
 			self.send_reply(Err.MemberIsSuspended, trid)
 			self.close()
 			return
-		if not user.verified_to_login:
-			self.send_reply(Err.AccountNotVerified, trid)
-			self.close()
-			return
+		#if not user.verified_to_login:
+		#	self.send_reply(Err.AccountNotVerified, trid)
+		#	self.close()
+		#	return
 		else:
-			self.send_reply('USR', trid, 'OK', user.email.lower(), *args)
+			print(f"args: {args}")
+			self.send_reply('USR', trid, 'OK', email, *args)
 		
 			(high, low) = uuid_to_high_low(user.uuid)
 			(ip, port) = self.peername
 			if dialect >= 19:
 				msg0 = encode_payload(PAYLOAD_MSG_0,
-					email_address = user.email,
+					email_address = email,
 					endpoint_ID = '{00000000-0000-0000-0000-000000000000}',
 					timestamp = now.isoformat()[:19] + 'Z',
 				)
@@ -482,20 +529,19 @@ class MSNPCtrlNS(MSNPCtrl):
 				self.send_reply('SBS', 0, 'null')
 				
 			msg1 = encode_payload(PAYLOAD_MSG_1,
-				time = int(now.timestamp()), high = high, low = low, email = user.email,
+				time = int(now.timestamp()), high = high, low = low, email = email,
 				token = token, ip = ip, port = port,
 				mpop = (0 if not machineguid else 1), nickname = user.status.name, server_ip = settings.TARGET_IP, sess_id = str(id(bs))
 			)
-			with open('config/restricted-emails.json', 'r') as file:
-				restricted_emails = json.load(file)
-				email_domain = email.lower().split('@')[-1]
-				if email_domain in restricted_emails or email_domain in disposable_emails:			
-					self.send_reply('NOT', encode_payload(PAYLOAD_MSG_10,
-						email = email, icon = '', url = 'https://crosstalk.im/alias-prep',
-						msg = "Your e-mail address contains a blacklisted domain and needs to be changed before you can log in. Click this alert to learn more. Change your e-mail in your CrossTalk account settings.",
-					))
-					self.close()
-					return
+
+			real_email_domain = user.email.lower().split('@')[-1]
+			if real_email_domain in restricted_emails or real_email_domain in disposable_emails:
+				self.send_reply('NOT', encode_payload(PAYLOAD_MSG_10,
+					email = email, icon = '', url = 'https://crosstalk.im/alias-prep',
+					msg = "Your e-mail address contains a blacklisted domain and needs to be changed before you can log in. Click this alert to learn more. Change your e-mail in your CrossTalk account settings.",
+				))
+				self.close()
+				return
 
 			if dialect >= 3:
 				self.send_reply('MSG', 'Hotmail', 'Hotmail', msg1)
@@ -519,9 +565,9 @@ class MSNPCtrlNS(MSNPCtrl):
 				#
 				# This seems to suggest data *was* sent in the initial UBX at some point (along with a status right before) - possibly related to MPoP. If we had more info to work with this could be possibly implemented.
 				if dialect >= 18:
-					rst = ('1:' + user.email.lower(),)  # single-element tuple
+					rst = ('1:' + email,)
 				else:
-					rst = (user.email.lower(), '1')
+					rst = (email, '1')
 				self.send_reply('UBX', *rst, b'')
 
 	# State = Live
@@ -557,7 +603,7 @@ class MSNPCtrlNS(MSNPCtrl):
 					cs = [c for c in contacts.values() if c.lists & lst]
 					if cs:
 						for i, c in enumerate(cs):
-							self.send_reply('LST', trid, lst.name, ser, i + 1, len(cs), c.head.email, c.status.name)
+							self.send_reply('LST', trid, lst.name, ser, i + 1, len(cs), self.backend.util_get_display_email(c.head), c.status.name)
 							if dialect >= 5:
 								for bpr_setting in ('PHH','PHM','PHW','MOB'):
 									bpr_value = c.head.settings.get(bpr_setting)
@@ -582,7 +628,7 @@ class MSNPCtrlNS(MSNPCtrl):
 					if cs:
 						for i, c in enumerate(cs):
 							gs = ((','.join([group.id for group in c._groups.copy()]) or '0') if lst == ContactList.FL else None)
-							self.send_reply('LST', trid, lst.name, ser, i + 1, len(cs), c.head.email, c.status.name or c.head.email, gs)
+							self.send_reply('LST', trid, lst.name, ser, i + 1, len(cs), self.backend.util_get_display_email(c.head), c.status.name or self.backend.util_get_display_email(c.head), gs)
 							for bpr_setting in ('PHH','PHM','PHW','MOB'):
 								bpr_value = c.head.settings.get(bpr_setting)
 								if bpr_value:
@@ -603,7 +649,7 @@ class MSNPCtrlNS(MSNPCtrl):
 					self.send_reply('LSG', g.id, g.name, 0)
 				for c in contacts.values():
 					self.send_reply(
-						'LST', c.head.email, c.status.name or c.head.email, int(c.lists),
+						'LST', self.backend.util_get_display_email(c.head), c.status.name or self.backend.util_get_display_email(c.head), int(c.lists),
 						','.join([group.id for group in c._groups.copy()]) or '0',
 					)
 					for bpr_setting in ('PHH','PHM','PHW','MOB'):
@@ -629,7 +675,7 @@ class MSNPCtrlNS(MSNPCtrl):
 					lists |= ContactList.PL
 					if lists & ContactList.RL:
 						lists &= ~ContactList.RL
-				self.send_reply('LST', 'N={}'.format(c.head.email), 'F={}'.format(c.status.name or c.head.email), ('C={}'.format(c.head.uuid) if c.lists & ContactList.FL else None),
+				self.send_reply('LST', 'N={}'.format(self.backend.util_get_display_email(c.head)), 'F={}'.format(c.status.name or self.backend.util_get_display_email(c.head)), ('C={}'.format(c.head.uuid) if c.lists & ContactList.FL else None),
 					int(lists), (None if dialect < 12 else '1'), ','.join([group.uuid for group in c._groups.copy()])
 				)
 				for bpr_setting in ('PHH','PHM','PHW','MOB'):
@@ -752,7 +798,6 @@ class MSNPCtrlNS(MSNPCtrl):
 		
 		if group_id == 'New%20Group':
 			# Bug: MSN 7.0 sends name instead of id in a particular scenario
-			
 			for g in detail._groups_by_id.values():
 				if g.name != 'New Group': continue
 				group_id = (g.id if self.dialect < 10 else g.uuid)
@@ -923,7 +968,7 @@ class MSNPCtrlNS(MSNPCtrl):
 							self.send_reply(Err.InvalidCircleMembership, trid)
 							return
 						membership = circle.memberships.get(user.uuid)
-						if membership is None or (membership is not None and membership.state != CircleState.Accepted) and self.dialect < 20: # hack until i figure out how to make WLM 2011 not dc when you send it 933:
+						if membership is None or (membership is not None and membership.state != CircleState.Accepted) and self.dialect < 20: # hack until i figure out how to make WLM 2011 not dc when you send it 933
 							self.send_reply(Err.InvalidCircleMembership, trid)
 							return
 					
@@ -944,7 +989,6 @@ class MSNPCtrlNS(MSNPCtrl):
 					ctc = None
 					lsts = None
 					
-					#networkid = NetworkID(int(c_el.get('t')))
 					username = c_el.get('n')
 					
 					if self.dialect >= 19:
@@ -1130,7 +1174,7 @@ class MSNPCtrlNS(MSNPCtrl):
 									lsts = ContactList(int(s_el.get('l')))
 							if lsts is None: continue
 						
-						contact_uuid = self.backend.util_get_uuid_from_email(email)
+						contact_uuid = backend.util_get_uuid_from_email(email)
 						
 						if contact_uuid is not None:
 							try:
@@ -1205,6 +1249,7 @@ class MSNPCtrlNS(MSNPCtrl):
 				self.send_reply(Err.InvalidUser, trid)
 			return
 		
+		
 		ctc_old = detail.contacts.get(contact_uuid)
 		ctc_old_lists = None
 		
@@ -1224,6 +1269,7 @@ class MSNPCtrlNS(MSNPCtrl):
 			return
 		
 		ser = self._ser()
+		ctc_head_email = self.backend.util_get_display_email(ctc_head)
 		
 		if dialect >= 10:
 			if lst == ContactList.FL:
@@ -1231,13 +1277,13 @@ class MSNPCtrlNS(MSNPCtrl):
 					self.send_reply('ADC', trid, lst_name, 'C={}'.format(ctc_head.uuid), group_id)
 				else:
 					self.send_reply(
-						'ADC', trid, lst_name, 'N={}'.format(ctc_head.email),
+						'ADC', trid, lst_name, 'N={}'.format(ctc_head_email),
 						('F={}'.format(ctc.status.name) if ctc.status.name else None), 'C={}'.format(ctc_head.uuid),
 					)
 			else:
-				self.send_reply('ADC', trid, lst_name, 'N={}'.format(ctc_head.email))
+				self.send_reply('ADC', trid, lst_name, 'N={}'.format(ctc_head_email))
 		else:
-			self.send_reply('ADD', trid, lst_name, ser, ctc_head.email, name, group_id)
+			self.send_reply('ADD', trid, lst_name, ser, ctc_head_email, name, group_id)
 		
 		if lst == ContactList.FL and (ctc_old_lists is not None and not ctc_old_lists & ContactList.FL):
 			if self.syn_sent and dialect >= 5:
@@ -1247,10 +1293,10 @@ class MSNPCtrlNS(MSNPCtrl):
 					if ctc_me is not None:
 						if ctc_me.lists & ContactList.AL:
 							send_bpr_info = True
-				self.send_reply('BPR', ser, ctc_head.email, 'PHH', ctc_head.settings.get('PHH') if send_bpr_info else None)
-				self.send_reply('BPR', ser, ctc_head.email, 'PHW', ctc_head.settings.get('PHW') if send_bpr_info else None)
-				self.send_reply('BPR', ser, ctc_head.email, 'PHM', ctc_head.settings.get('PHM') if send_bpr_info else None)
-				self.send_reply('BPR', ser, ctc_head.email, 'MOB', ctc_head.settings.get('MOB', 'N') if send_bpr_info else 'N')
+				self.send_reply('BPR', ser, ctc_head_email, 'PHH', ctc_head.settings.get('PHH') if send_bpr_info else None)
+				self.send_reply('BPR', ser, ctc_head_email, 'PHW', ctc_head.settings.get('PHW') if send_bpr_info else None)
+				self.send_reply('BPR', ser, ctc_head_email, 'PHM', ctc_head.settings.get('PHM') if send_bpr_info else None)
+				self.send_reply('BPR', ser, ctc_head_email, 'MOB', ctc_head.settings.get('MOB', 'N') if send_bpr_info else 'N')
 			
 			bs.evt.on_presence_notification(ctc, False, Substatus.Offline, trid = trid, updated_phone_info = {
 				'PHH': ctc_head.settings.get('PHH'),
@@ -1340,7 +1386,7 @@ class MSNPCtrlNS(MSNPCtrl):
 		
 		if settings.FORCE_HOSTS_UPDATE:
 			self.send_reply('NOT', encode_payload(PAYLOAD_MSG_10,
-				email = user.email, icon = '', url = 'http://diet.crosstalk.im/moving-hosts',
+				email = self.backend.util_get_display_email(user), icon = '', url = 'http://diet.crosstalk.im/moving-hosts',
 				msg = "Your patches need to be updated before you can log in. Click this alert for more info."
 			))
 			self.backend.loop.call_later(5, self.close)
@@ -1398,8 +1444,6 @@ class MSNPCtrlNS(MSNPCtrl):
 						self.send_reply(*m)
 			if dialect >= 6 and not self.is_third_party_client:
 				self._send_chl(trid)
-			#	msg = encode_payload(PAYLOAD_MSG_11, unread = random.randrange(1,300))
-			#	self.send_reply('MSG', 'Hotmail', 'Hotmail', msg)
 			if dialect >= 11:
 				msg2 = encode_payload(PAYLOAD_MSG_2,
 					ct = 'text/x-msmsgsinitialmdatanotification', md = gen_mail_data(user, backend),
@@ -1417,12 +1461,7 @@ class MSNPCtrlNS(MSNPCtrl):
 	def _m_qry(self, trid: str, client_id: str, response: bytes) -> None:
 		challenge = self.challenge
 		
-		if not challenge:
-			self.send_reply(Err.ChallengeResponseFailed, trid)
-			self.close()
-			return
-		
-		id_key, max_dialect, skip_challenge = _QRY_KEYPAIRS[client_id]
+		id_key, max_dialect = _QRY_KEYPAIRS[client_id]
 		
 		if self.dialect > max_dialect:
 			self.send_reply(Err.ChallengeResponseFailed, trid)
@@ -1433,7 +1472,7 @@ class MSNPCtrlNS(MSNPCtrl):
 		if settings.DEBUG_FULL:
 			print(f"Server challenge response: {server_response}")
 		
-		if not skip_challenge and response.decode() != server_response:
+		if response.decode() != server_response:
 			self.send_reply(Err.ChallengeResponseFailed, trid)
 			self.close()
 			return
@@ -1474,9 +1513,7 @@ class MSNPCtrlNS(MSNPCtrl):
 			chat_id = circle_id[-12:]
 		
 		nfy_1_index = data.index(b'\r\n\r\n', i) + 4
-		
-		#nfy_delivery = data[i:nfy_1_index].decode('utf-8')
-		
+				
 		# TODO: `PUT` ACK
 		
 		nfy_actual = data[nfy_1_index:]
@@ -1484,8 +1521,6 @@ class MSNPCtrlNS(MSNPCtrl):
 		payload_index = nfy_actual.index(b'\r\n\r\n') + 4
 		nfy_headers = Parser().parsestr(nfy_actual[:payload_index].decode('utf-8'))
 		payload = nfy_actual[payload_index:]
-		#header_end = nfy_actual.find('\r\n\r\n')
-		#other_headers = nfy_actual[header_end+4:payload_index].split('\r\n\r\n')
 		
 		if nfy_headers.get('Content-Type') == 'application/circles+xml':
 			if not self.circle_adl_sent:
@@ -1509,7 +1544,8 @@ class MSNPCtrlNS(MSNPCtrl):
 					self.send_reply(Err.InvalidParameter, trid)
 					return
 				email = email.split('1:', 1)[1]
-				if email == user.email:
+				user_display_email = self.backend.util_get_display_email(user)
+				if email.lower() in (user.email.lower(), user_display_email.lower()):
 					cs = backend.get_circle_cs(chat_id, bs)
 					if cs is None:
 						self.send_reply(Err.InvalidParameter, trid)
@@ -1539,7 +1575,8 @@ class MSNPCtrlNS(MSNPCtrl):
 			try:
 				payload_xml = parse_xml(payload)
 				
-				if not (to[1] is NetworkID.WINDOWS_LIVE and to[0] == user.email and to[2] is None):
+				user_display_email = self.backend.util_get_display_email(user)
+				if not (to[1] is NetworkID.WINDOWS_LIVE and to[0].lower() in (user.email.lower(), user_display_email.lower()) and to[2] is None):
 					return
 				
 				name = None # type: Optional[str]
@@ -1648,7 +1685,7 @@ class MSNPCtrlNS(MSNPCtrl):
 				
 				#TODO: Presence is a bit wonky
 				bs.me_update({
-					'name': name or user.email,
+					'name': name or user_display_email,
 					'message': psm,
 					'substatus': substatus,
 					'media': currentmedia,
@@ -1686,7 +1723,6 @@ class MSNPCtrlNS(MSNPCtrl):
 		circle_mode = False
 		chat_id = None
 		cs = None
-		presence = False
 
 		try:
 			i = data.index(b'\r\n\r\n') + 4
@@ -1751,7 +1787,6 @@ class MSNPCtrlNS(MSNPCtrl):
 					self.send_reply(Err.GetCodeForException(ex, self.dialect), trid)
 					return
 			else:
-				# TODO 
 				pass
 
 	def _m_rea(self, trid: str, email: str, name: str) -> None:
@@ -1763,7 +1798,8 @@ class MSNPCtrlNS(MSNPCtrl):
 		bs = self.bs
 		assert bs is not None
 		
-		if email == bs.user.email:
+		user_display_email = self.backend.util_get_display_email(bs.user)
+		if email.lower() in (bs.user.email.lower(), user_display_email.lower()):
 			bs.me_update({ 'name': name })
 		self.send_reply('REA', trid, self._ser(), email, name)
 	
@@ -1775,7 +1811,7 @@ class MSNPCtrlNS(MSNPCtrl):
 		if self.dialect >= 5:
 			self.send_reply(Err.CommandDisabled, trid)
 
-		sender_email = bs.user.email
+		sender_email = self.backend.util_get_display_email(bs.user)
 
 		bs.send_invitation_email(email, sender_email, None, None)
 		self.send_reply('SND', trid, 'OK')
@@ -1789,7 +1825,7 @@ class MSNPCtrlNS(MSNPCtrl):
 			self.close()
 			return
 
-		sender_email = bs.user.email
+		sender_email = self.backend.util_get_display_email(bs.user)
 
 		bs.send_invitation_email(email, sender_email, name, message)
 		self.send_reply('SDC', trid, 'OK')
@@ -1803,12 +1839,11 @@ class MSNPCtrlNS(MSNPCtrl):
 		return
 
 	def _m_del(self, trid: str, data: bytes) -> None:
-		# OUT sends presnece updates on logout, ignore for now until chatting is supported
+		# OUT sends presence updates on logout, ignore for now until we have multiparties implemented
 		pass
 	
 	def _m_prp(self, trid: str, key: str, value: Optional[str] = None, *rest: Optional[str]) -> None:
 		# Set a display name
-		#>>> PRP 115 MFN ~~woot~~
 		dialect = self.dialect
 		if dialect < 5:
 			self.close()
@@ -1910,7 +1945,7 @@ class MSNPCtrlNS(MSNPCtrl):
 	
 	def _m_fqy(self, trid: str, data: bytes) -> None:
 		# "Federated query; Query contact's network types"
-		# https://web.archive.org/web/20100820020114/http://msnpiki.msnfanatic.com:80/index.php/Command:FQY
+		# https://yellows111.darkok.xyz/msnp/commands/fqy.html
 		d_els = None
 		domain = None
 		username = None
@@ -1956,7 +1991,7 @@ class MSNPCtrlNS(MSNPCtrl):
 	
 	def _m_uun(self, trid: str, email: str, type: str, data: Optional[bytes] = None) -> None:
 		# "Send sharing invitation or reply to invitation"
-		# https://web.archive.org/web/20130926060507/http://msnpiki.msnfanatic.com/index.php/MSNP13:Changes#UUN
+		# https://yellows111.darkok.xyz/msnp/commands/uun.html
 		if self.dialect < 13:
 			self.close()
 			return
@@ -2004,7 +2039,6 @@ class MSNPCtrlNS(MSNPCtrl):
 		pop_id_self = bs.front_data.get('msn_pop_id')
 		
 		for sess_notify in self.backend.util_get_sessions_by_user(ctc_head):
-			#if sess_notify is self: continue
 			sess_notify.evt.msn_on_uun_sent(bs.user, uun_type, data, pop_id_sender = pop_id_self, pop_id = pop_id)
 		
 		self.send_reply('UUN', trid, 'OK')
@@ -2160,7 +2194,7 @@ class BackendEventHandler(event.BackendEventHandler):
 		user = bs.user
 
 		self.ctrl.send_reply('NOT', encode_payload(PAYLOAD_MSG_10,
-			email = user.email, icon = icon_url, url = url,
+			email = self.ctrl.backend.util_get_display_email(user), icon = icon_url, url = url,
 			msg = message,
 		))
 
@@ -2194,7 +2228,7 @@ class BackendEventHandler(event.BackendEventHandler):
 		if 5 <= self.ctrl.dialect < 13 and updated_phone_info and self.ctrl.syn_sent:
 			for phone_type, value in updated_phone_info.items():
 				if value is not None:
-					self.ctrl.send_reply('BPR', self.ctrl._ser(), ctc.head.email, phone_type, None if send_status_on_bl else value)
+					self.ctrl.send_reply('BPR', self.ctrl._ser(), self.ctrl.backend.util_get_display_email(ctc.head), phone_type, None if send_status_on_bl else value)
 		
 		if update_status or update_info_other:
 			for m in build_presence_notif(
@@ -2240,7 +2274,7 @@ class BackendEventHandler(event.BackendEventHandler):
 					extra += (1,)
 				token, _ = self.ctrl.backend.auth_service.create_token('sb/cal', (self.ctrl.bs, dialect, chat), lifetime = 120)
 				self.ctrl.send_reply(
-					'RNG', chat.ids['main'], f'{settings.TARGET_IP}:1865', 'CKI', token, inviter.email, inviter.status.name, *extra,
+					'RNG', chat.ids['main'], f'{settings.TARGET_IP}:1865', 'CKI', token, self.ctrl.backend.util_get_display_email(inviter), inviter.status.name, *extra,
 				)
 	
 	def on_declined_chat_invite(self, chat: Chat, circle: bool = False) -> None:
@@ -2248,7 +2282,7 @@ class BackendEventHandler(event.BackendEventHandler):
 			self.msn_on_notify_ab()
 	
 	def on_added_me(self, user: User, *, adder_id: Optional[str] = None, message: Optional[TextWithData] = None) -> None:
-		email = user.email
+		email = self.ctrl.backend.util_get_display_email(user)
 		name = (user.status.name or email)
 		dialect = self.ctrl.dialect
 		bs = self.ctrl.bs
@@ -2278,7 +2312,7 @@ class BackendEventHandler(event.BackendEventHandler):
 			self.msn_on_notify_ab()
 	
 	def on_removed_me(self, user: User) -> None:
-		email = user.email
+		email = self.ctrl.backend.util_get_display_email(user)
 		dialect = self.ctrl.dialect
 		bs = self.ctrl.bs
 		assert bs is not None
@@ -2331,9 +2365,9 @@ class BackendEventHandler(event.BackendEventHandler):
 			if normalize_pop_id(pop_id).lower() != pop_id_self.lower(): return
 		
 		if pop_id_sender is not None and pop_id is not None and ctrl.dialect >= 16:
-			email = '{};{}'.format(sender.email, '{' + pop_id_sender + '}')
+			email = '{};{}'.format(self.ctrl.backend.util_get_display_email(sender), '{' + pop_id_sender + '}')
 		else:
-			email = sender.email
+			email = self.ctrl.backend.util_get_display_email(sender)
 		
 		self.ctrl.send_reply('UBN', email, type, data)
 	
@@ -2345,7 +2379,7 @@ class BackendEventHandler(event.BackendEventHandler):
 		id_bits = uuid_to_high_low(user.uuid)
 		self.ctrl.send_reply('NOT', encode_payload(PAYLOAD_MSG_4,
 			member_low = binascii.hexlify(struct.pack('!I', id_bits[1])).decode('utf-8'),
-			member_high = binascii.hexlify(struct.pack('!I', id_bits[0])).decode('utf-8'), email = user.email,
+			member_high = binascii.hexlify(struct.pack('!I', id_bits[0])).decode('utf-8'), email = self.ctrl.backend.util_get_display_email(user),
 			cid = cid_format(user.uuid, decimal = True), now = date_format(datetime.utcnow()),
 		))
 	
@@ -2358,7 +2392,7 @@ class BackendEventHandler(event.BackendEventHandler):
 		
 		self.ctrl.send_reply('NOT', encode_payload(PAYLOAD_MSG_8,
 			member_low = binascii.hexlify(struct.pack('!I', id_bits[1])).decode('utf-8'),
-			member_high = binascii.hexlify(struct.pack('!I', id_bits[0])).decode('utf-8'), email = user.email,
+			member_high = binascii.hexlify(struct.pack('!I', id_bits[0])).decode('utf-8'), email = self.ctrl.backend.util_get_display_email(user),
 			chat_id = chat_id,
 		))
 	
@@ -2453,14 +2487,13 @@ class ChatEventHandler(event.ChatEventHandler):
 		
 		chat = self.cs.chat
 		circle = chat.circle
-		#assert circle is not None
 		
 		membership = circle.memberships[user.uuid]
 		if membership.state == CircleState.Empty:
 			for cs_other in chat.get_roster():
 				self.ctrl.send_reply('NFY', 'DEL', encode_payload(PAYLOAD_MSG_9,
-					to_email = user.email, nid = str(int(NetworkID.CIRCLE)), uuid = '00000000-0000-0000-0009-{}'.format(circle.chat_id),
-					from_email = cs_other.user.email,
+					to_email = self.ctrl.backend.util_get_display_email(user), nid = str(int(NetworkID.CIRCLE)), uuid = '00000000-0000-0000-0009-{}'.format(circle.chat_id),
+					from_email = self.ctrl.backend.util_get_display_email(cs_other.user),
 				))
 	
 	def on_participant_joined(self, cs_other: ChatSession, first_pop: bool, initial_join: bool) -> None:
@@ -2471,7 +2504,6 @@ class ChatEventHandler(event.ChatEventHandler):
 		cs = self.cs
 		chat = cs.chat
 		circle = chat.circle
-		#assert circle is not None
 		if circle:
 			if (cs_other.user.status.substatus is Substatus.Invisible and cs_other.user is user) or not cs_other.user.status.is_offlineish():
 				if cs_other.user.uuid == circle.owner_uuid:
@@ -2499,15 +2531,14 @@ class ChatEventHandler(event.ChatEventHandler):
 		cs = self.cs
 		chat = cs.chat
 		circle = chat.circle
-		#assert circle is not None
 		
 		if last_pop:
 			if circle:
 				membership = circle.memberships[cs_other.user.uuid]
 				if membership.state == CircleState.Empty:
 					self.ctrl.send_reply('NFY', 'DEL', encode_payload(PAYLOAD_MSG_9,
-						to_email = user.email, nid = str(int(NetworkID.CIRCLE)), uuid = '00000000-0000-0000-0009-{}'.format(circle.chat_id),
-						from_email = cs_other.user.email,
+						to_email = self.ctrl.backend.util_get_display_email(user), nid = str(int(NetworkID.CIRCLE)), uuid = '00000000-0000-0000-0009-{}'.format(circle.chat_id),
+						from_email = self.ctrl.backend.util_get_display_email(cs_other.user),
 					))
 	
 	def on_chat_invite_declined(
@@ -2530,7 +2561,6 @@ class ChatEventHandler(event.ChatEventHandler):
 		
 		chat = self.cs.chat
 		circle = chat.circle
-		#assert circle is not None
 		
 		presence = CIRCLE_PROPS.format(
 			friendly = circle.name, psm = chat.front_data.get('msn_circle_psm') or '', cm = chat.front_data.get('msn_circle_cm') or '',
@@ -2539,7 +2569,7 @@ class ChatEventHandler(event.ChatEventHandler):
 		result = CIRCLE.format(presence)
 		
 		self.ctrl.send_reply('NFY', 'PUT', encode_payload(PAYLOAD_MSG_6,
-			email = _encode_email_epid(user.email, bs.front_data.get('msn_pop_id')), chat_id = circle.chat_id,
+			email = _encode_email_epid(self.ctrl.backend.util_get_display_email(user), bs.front_data.get('msn_pop_id')), chat_id = circle.chat_id,
 			cl = len(result), payload = result,
 		))
 	
@@ -2551,16 +2581,15 @@ class ChatEventHandler(event.ChatEventHandler):
 		cs = self.cs
 		chat = cs.chat
 		circle = chat.circle
-		#assert circle is not None
 		
-		users = _get_circle_roster(chat, cs)
+		users = _get_circle_roster(self, chat, cs)
 			
 		roster = CIRCLE_ROSTER.format(users = users)
 		
 		result = CIRCLE.format(roster)
 		
 		self.ctrl.send_reply('NFY', 'PUT', encode_payload(PAYLOAD_MSG_5,
-			email = user.email, chat_id = circle.chat_id,
+			email = self.ctrl.backend.util_get_display_email(user), chat_id = circle.chat_id,
 			cl = len(result), payload = result,
 		))
 	
@@ -2572,7 +2601,6 @@ class ChatEventHandler(event.ChatEventHandler):
 		cs = self.cs
 		chat = cs.chat
 		circle = chat.circle
-		#assert circle is not None
 		
 		membership = circle.memberships[cs_other.user.uuid]
 		if membership.state == CircleState.Empty: return
@@ -2629,11 +2657,11 @@ class ChatEventHandler(event.ChatEventHandler):
 				return True
 		return False
 
-def _get_circle_roster(chat: Chat, cs: ChatSession) -> str:
+def _get_circle_roster(self, chat: Chat, cs: ChatSession) -> str:
 	circle = chat.circle
 	assert circle is not None
 
-	users = ''.join(CIRCLE_USER.format(email=cs1.user.email) for cs1 in chat.get_roster_single() 
+	users = ''.join(CIRCLE_USER.format(email=self.ctrl.backend.util_get_display_email(cs1.user)) for cs1 in chat.get_roster_single() 
 					if not (cs1.user.status.is_offlineish() or circle.memberships[cs1.user.uuid].blocking) 
 					or cs1.user is not cs.user)
 	
@@ -2690,9 +2718,8 @@ def messagedata_from_sdg(sender: User, sender_pop_id: Optional[str], data: bytes
 		type = type_mapping.get(message_type_str, MessageType.Chat)
 		text = body_raw.decode('utf-8') if type == MessageType.Chat or MessageType.MSNP2P else ''
 	except Exception:
-		#type = MessageType.Chat
-		#text = "CrossTalk System Message: This message cannot be viewed on the client you're currently using."
-		pass
+		type = MessageType.Chat
+		text = "CrossTalk System Message: This message cannot be viewed on the client you're currently using."
 
 	message = MessageData(sender=sender, sender_pop_id=sender_pop_id, type=type, text=text)
 	message.front_cache['msnp_sdg'] = data
@@ -2706,7 +2733,7 @@ def messagedata_to_sdg(self, data: MessageData, user: User, *, circle: Optional[
 				s = (
 					f'Content-Length: 2\r\nContent-Type: text/x-msmsgscontrol\r\n'
 					f'Content-Transfer-Encoding: 7bit\r\nMessage-Type: Control\r\n'
-					f'Message-Subtype: Typing\r\nMIME-Version: 1.0\r\nTypingUser: {data.sender.email}\r\n\r\n\r\n'
+					f'Message-Subtype: Typing\r\nMIME-Version: 1.0\r\nTypingUser: {self.ctrl.backend.util_get_display_email(data.sender)}\r\n\r\n\r\n'
 				).encode('utf-8')
 			else:
 				s = 'Message-Type: Control/Typing\r\nContent-Length: 0\r\n\r\n'.encode('utf-8')
@@ -2740,7 +2767,10 @@ def messagedata_to_sdg(self, data: MessageData, user: User, *, circle: Optional[
 		else:
 			raise ValueError("Unknown message type", data.type)
 
-		to = f'9:00000000-0000-0000-0009-{circle.chat_id}@live.com;path=IM' if circle else f'1:{_encode_email_epid(user.email, self.bs.front_data.get('msn_pop_id'))}'
+		sender_display_email = self.ctrl.backend.util_get_display_email(data.sender)
+		user_display_email = self.ctrl.backend.util_get_display_email(user)
+
+		to = f'9:00000000-0000-0000-0009-{circle.chat_id}@live.com;path=IM' if circle else f'1:{_encode_email_epid(user_display_email, self.bs.front_data.get('msn_pop_id'))}'
 		pre = f'Routing: 1.0\r\nTo: {to}\r\n'
 
 		if data.type == MessageType.MSNP2P:
@@ -2750,7 +2780,7 @@ def messagedata_to_sdg(self, data: MessageData, user: User, *, circle: Optional[
 
 		timestamp = datetime.now(timezone.utc).strftime('%Y-%m-%dT%H:%M:%S.%f')[:-3] + 'Z'
 		pre += f'Original-Arrival-Time: {timestamp}\r\n'
-		pre += f'From: 1:{_encode_email_epid(data.sender.email, data.sender_pop_id)}\r\n\r\n'
+		pre += f'From: 1:{_encode_email_epid(sender_display_email, data.sender_pop_id)}\r\n\r\n'
 		pre = pre.encode('utf-8')
 
 		r = 'Reliability: 1.0'
@@ -2967,44 +2997,44 @@ TIMESTAMP = '2000-01-01T00:00:00.0-00:00'
 
 _QRY_KEYPAIRS = {
 	# MSNP6 - 9
-	'msmsgs@msnmsgr.com': ('Q1P7W2E4J9R8U3S5', 10, False),
-	'msntv@msnmsgr.com': ('<unknown>', 10, True), # found in MSN TV 2.7 build 16109
-	'macmsgr@msnmsgr.com': ('A8J3D5F7L3K2V6F4', 10, False),
-	'assist@msnmsgr.com': ('L2P3B7C6V9J4T8D5', 10, False),
-	'appshare@msnmsgr.com': ('W5N2C9D7A6P3K4J8', 10, False),
-	'pocketpc@msnmsgr.com': ('<unknown>', 10, True),
-	'TEST00578M1WM}RR': ('<unknown>', 10, True),
-	'PROD0038W!61ZTF9': ('VT6PX?UQTM4WM%YR', 10, True),
-	'PROD0058#7IL2{QD': ('QHDCY@7R1TB6W?5B', 10, False),
-	'PROD0061VRRZH@4F': ('JXQ6J@TUOGYV@N0M', 10, False),
-	'PROD0062I2RVG#RV': ('LPOFJ{8L6AM2N!G', 10, False),
-	'PROD0066X_86JBY8': ('%_IP#M2WDG247}@I', 10, False),
-	'PROD00678K?EPLA}': ('MP8JK{DUDLDMC{9L', 10, False),
-	'PROD0074Z}QA4HPI': ('5JHDY@F5_KLEF?3O', 10, False),
-	'PROD0075THRTM{7!': ('WLJIQ$8LDLNI_J4Q', 10, False),
-	'PROD0076ENE8*@AW': ('CEQJ8}OE0!WTSWII', 10, False),
-	'PROD0039E3VGM%GB': ('B7WRX$T9S3875{68', 10, False), # found in MSN Messenger 4.7.0106
-	'PROD00504RLUG%WL': ('I2EBK%PYNLZL5_J4', 10, False),
-	'PROD00517IFH4@RV': ('MYRED!3QTCFWG@9G', 10, False), # found in Windows Messenger 5.1.0701
-	'PROD00444_M6XYJT': ('UMJBL@QN17VEI{5L', 10, False), # found in Messenger for Mac 3.0
-	'PROD0045YI56T?TX': ('FV!WOP5UKXO8$LV$', 10, False),
-	'PROD0046K9O#QFXY': ('8{B7#LEX_V5HV@SQ', 10, False),
+	'msmsgs@msnmsgr.com': ('Q1P7W2E4J9R8U3S5', 10),
+	'msntv@msnmsgr.com': ('<unknown>', 10), # found in MSN TV 2.7 build 16109
+	'macmsgr@msnmsgr.com': ('A8J3D5F7L3K2V6F4', 10),
+	'assist@msnmsgr.com': ('L2P3B7C6V9J4T8D5', 10),
+	'appshare@msnmsgr.com': ('W5N2C9D7A6P3K4J8', 10),
+	'pocketpc@msnmsgr.com': ('<unknown>', 10),
+	'TEST00578M1WM}RR': ('<unknown>', 10),
+	'PROD0038W!61ZTF9': ('VT6PX?UQTM4WM%YR', 10),
+	'PROD0058#7IL2{QD': ('QHDCY@7R1TB6W?5B', 10),
+	'PROD0061VRRZH@4F': ('JXQ6J@TUOGYV@N0M', 10),
+	'PROD0062I2RVG#RV': ('LPOFJ{8L6AM2N!G', 10),
+	'PROD0066X_86JBY8': ('%_IP#M2WDG247}@I', 10),
+	'PROD00678K?EPLA}': ('MP8JK{DUDLDMC{9L', 10),
+	'PROD0074Z}QA4HPI': ('5JHDY@F5_KLEF?3O', 10),
+	'PROD0075THRTM{7!': ('WLJIQ$8LDLNI_J4Q', 10),
+	'PROD0076ENE8*@AW': ('CEQJ8}OE0!WTSWII', 10),
+	'PROD0039E3VGM%GB': ('B7WRX$T9S3875{68', 10), # found in MSN Messenger 4.7.0106
+	'PROD00504RLUG%WL': ('I2EBK%PYNLZL5_J4', 10),
+	'PROD00517IFH4@RV': ('MYRED!3QTCFWG@9G', 10), # found in Windows Messenger 5.1.0701
+	'PROD00444_M6XYJT': ('UMJBL@QN17VEI{5L', 10), # found in Messenger for Mac 3.0
+	'PROD0045YI56T?TX': ('FV!WOP5UKXO8$LV$', 10),
+	'PROD0046K9O#QFXY': ('8{B7#LEX_V5HV@SQ', 10),
 	# MSNP11 - 12
-	'PROD008955JTJ_S7': ('DHCPQ$8JI5HD3{4L', 12, False),
-	'PROD0090YUAUV{2B': ('YMM8C_H7KCQ2S_KL', 12, False),
-	'PROD00974#MT*RC2': ('LMCVO*18PQJ3H!K3', 12, False),
-	'PROD0101{0RM?UBW': ('CFHUR$52U_{VIX5T', 12, True),
-	'PROD0102LUNTP%M?': ('JD5QT%#ILEBP5?LI', 12, False),
-	"PROD0104U6VVM{UJ": ('VK67B}379XYM5}$T', 12, False),
+	'PROD008955JTJ_S7': ('DHCPQ$8JI5HD3{4L', 12),
+	'PROD0090YUAUV{2B': ('YMM8C_H7KCQ2S_KL', 12),
+	'PROD00974#MT*RC2': ('LMCVO*18PQJ3H!K3', 12),
+	'PROD0101{0RM?UBW': ('CFHUR$52U_{VIX5T', 12),
+	'PROD0102LUNTP%M?': ('JD5QT%#ILEBP5?LI', 12),
+	"PROD0104U6VVM{UJ": ('VK67B}379XYM5}$T', 12),
 	# MSNP13 - 14
-	'PROD01065C%ZFN6F': ('O4BG@C7BWLYQX?5G', 14, True),
-	'PROD0112J1LW7%NB': ('RH96F{PHI8PPX_TJ', 14, False),
+	'PROD01065C%ZFN6F': ('O4BG@C7BWLYQX?5G', 14),
+	'PROD0112J1LW7%NB': ('RH96F{PHI8PPX_TJ', 14),
 	# MSNP15+
-	'PROD0113H11T8$X_': ('RG@XY*28Q5QHS%Q5', 21, True),
-	'PROD0114ES4Z%Q5W': ('PK}_A_0N_K%O?A9S', 21, True),
-	'PROD0116PE?TSI1': ('EXFK#_48PJR82_3G', 21, True),
-	'PROD0118R6%2WYOS': ('YIXPX@5I2P0UT*LK', 21, True),
-	'PROD0119GSJUC$18': ('ILTXC!4IXB5FB*PX', 21, True),
+	'PROD0113H11T8$X_': ('RG@XY*28Q5QHS%Q5', 21),
+	'PROD0114ES4Z%Q5W': ('PK}_A_0N_K%O?A9S', 21),
+	'PROD0116PE?TSI1': ('EXFK#_48PJR82_3G', 21),
+	'PROD0118R6%2WYOS': ('YIXPX@5I2P0UT*LK', 21),
+	'PROD0119GSJUC$18': ('ILTXC!4IXB5FB*PX', 21),
 	# Thanks J.M. for making the tweet with this WLM 2009 ID-key combo. ^_^
-	'PROD0120PW!CCV9@': ('C1BX{V4W}Q3*10SM', 21, True),
+	'PROD0120PW!CCV9@': ('C1BX{V4W}Q3*10SM', 21),
 }

front/msn/msnp_sb.py

@@ -46,6 +46,7 @@ class MSNPCtrlSB(MSNPCtrl):
 	# State = Auth
 	
 	def _m_usr(self, trid: Optional[str], arg: Optional[str], token: Optional[str], *args: Any) -> None:
+		backend = self.backend
 		#>>> USR trid email@example.com token (MSNP < 16)
 		#>>> USR trid email@example.com;{00000000-0000-0000-0000-000000000000} token (MSNP >= 16)
 		self.auth_sent = True
@@ -54,6 +55,9 @@ class MSNPCtrlSB(MSNPCtrl):
 			return
 		
 		(email, pop_id) = decode_email_pop(arg)
+
+		username, domain = email.split('@', 1)
+				
 		
 		data = self.backend.auth_service.pop_token('sb/xfr', token) # type: Optional[Tuple[BackendSession, int]]
 		if data is None:
@@ -63,7 +67,18 @@ class MSNPCtrlSB(MSNPCtrl):
 		
 		bs, dialect = data
 		bs_pop_id = bs.front_data.get('msn_pop_id') or ''
-		if bs.user.email.lower() != email.lower() or (dialect >= 16 and pop_id is not None and bs_pop_id.lower() != normalize_pop_id(pop_id).lower()):
+		if dialect == 2:
+			uuid = backend.user_service.get_uuid_username(username)
+			if uuid is None:
+				self.send_reply(Err.AuthFail, trid)
+				self.close()
+				return
+			user = backend.user_service.get(uuid)
+			email = user.email
+			
+		alias_email = '{}@crosstalk.im'.format(bs.user.username).lower()
+		valid_emails = (bs.user.email.lower(), alias_email)
+		if email.lower() not in valid_emails or (bs.user.alias_active and email.lower() != alias_email) or (dialect >= 16 and pop_id is not None and bs_pop_id.lower() != normalize_pop_id(pop_id).lower()):
 			self.send_reply(Err.AuthFail, trid)
 			self.close()
 			return
@@ -107,7 +122,9 @@ class MSNPCtrlSB(MSNPCtrl):
 		
 		(bs, dialect, chat) = data
 		bs_pop_id = bs.front_data.get('msn_pop_id') or ''
-		if bs.user.email.lower() != email or (dialect >= 16 and pop_id is not None and bs_pop_id.lower() != normalize_pop_id(pop_id).lower()):
+		alias_email = '{}@crosstalk.im'.format(bs.user.username).lower()
+		valid_emails = (bs.user.email.lower(), alias_email)
+		if email.lower() not in valid_emails or (bs.user.alias_active and email.lower() != alias_email) or (dialect >= 16 and pop_id is not None and bs_pop_id.lower() != normalize_pop_id(pop_id).lower()):
 			self.send_reply(Err.AuthFail, trid)
 			self.close()
 			return
@@ -153,17 +170,18 @@ class MSNPCtrlSB(MSNPCtrl):
 			i = 1
 			for other_cs in tmp:
 				other_user = other_cs.user
+				other_user_display_email = self.backend.util_get_display_email(other_user)
 				capabilities = encode_capabilities_capabilitiesex(
 					((other_cs.bs.front_data.get('msn_capabilities') or 0) if other_cs.bs.front_data.get('msn') is True else MAX_CAPABILITIES_BASIC),
 					other_cs.bs.front_data.get('msn_capabilitiesex') or 0,
 				)
 				self.send_reply(
-					'IRO', trid, i, l, encode_email_pop(other_user.email, other_cs.bs.front_data.get('msn_pop_id')),
+					'IRO', trid, i, l, encode_email_pop(other_user_display_email, other_cs.bs.front_data.get('msn_pop_id')),
 					other_user.status.name, capabilities,
 				)
 				if other_cs.primary_pop and other_cs.bs.front_data.get('msn_pop_id') is not None:
 					i += 1
-					self.send_reply('IRO', trid, i, l, other_user.email, other_user.status.name, capabilities)
+					self.send_reply('IRO', trid, i, l, other_user_display_email, other_user.status.name, capabilities)
 				i += 1
 		else:
 			roster_one_per_user = [] # type: List[ChatSession]
@@ -186,7 +204,7 @@ class MSNPCtrlSB(MSNPCtrl):
 							else MAX_CAPABILITIES_BASIC
 						),
 					)
-				self.send_reply('IRO', trid, i + 1, l, other_user.email, other_user.status.name, *extra)
+				self.send_reply('IRO', trid, i + 1, l, self.backend.util_get_display_email(other_user), other_user.status.name, *extra)
 		
 		self.send_reply('ANS', trid, 'OK')
 		
@@ -301,6 +319,7 @@ class ChatEventHandler(event.ChatEventHandler):
 		if (not first_pop or cs_other.user is cs.user) and ctrl.dialect < 16: return
 		
 		user = cs_other.user
+		user_display_email = ctrl.backend.util_get_display_email(user)
 		
 		pop_id_other = cs_other.bs.front_data.get('msn_pop_id')
 		if (
@@ -310,9 +329,9 @@ class ChatEventHandler(event.ChatEventHandler):
 			)
 			and ctrl.dialect >= 16
 		):
-			email = '{};{}'.format(user.email, '{' + pop_id_other + '}')
+			email = '{};{}'.format(user_display_email, '{' + pop_id_other + '}')
 		else:
-			email = user.email
+			email = user_display_email
 		
 		# Capability flags in JOI were technically introduced by MSNP13, but there are MSNP12 logs that show that they
 		# had been introduced in that protocol version by then
@@ -339,19 +358,20 @@ class ChatEventHandler(event.ChatEventHandler):
 			extra = ()
 		ctrl.send_reply('JOI', email, user.status.name, *extra)
 		if cs_other.user is not cs.user and pop_id_other is not None and first_pop and ctrl.dialect >= 16:
-			ctrl.send_reply('JOI', user.email, user.status.name, *extra)
+			ctrl.send_reply('JOI', user_display_email, user.status.name, *extra)
 	
 	def on_participant_left(self, cs_other: ChatSession, last_pop: bool) -> None:
 		ctrl = self.ctrl
 		if not last_pop and ctrl.dialect < 16: return
+		user_display_email = ctrl.backend.util_get_display_email(cs_other.user)
 		pop_id_other = cs_other.bs.front_data.get('msn_pop_id')
 		if pop_id_other is not None and ctrl.dialect >= 16:
-			email = '{};{}'.format(cs_other.user.email, '{' + pop_id_other + '}')
+			email = '{};{}'.format(user_display_email, '{' + pop_id_other + '}')
 		else:
-			email = cs_other.user.email
+			email = user_display_email
 		self.ctrl.send_reply('BYE', email)
 		if last_pop and pop_id_other is not None and ctrl.dialect >= 16:
-			self.ctrl.send_reply('BYE', cs_other.user.email)
+			self.ctrl.send_reply('BYE', user_display_email)
 	
 	def on_chat_invite_declined(
 		self, chat: Chat, invitee: User, *, invitee_id: Optional[str] = None, message: Optional[str] = None, circle: bool = False,
@@ -369,7 +389,7 @@ class ChatEventHandler(event.ChatEventHandler):
 	
 	def on_message(self, data: MessageData) -> None:
 		if data.type is not MessageType.TypingDone:
-			self.ctrl.send_reply('MSG', data.sender.email, data.sender.status.name, messagedata_to_msnp(data))
+			self.ctrl.send_reply('MSG', self.ctrl.backend.util_get_display_email(data.sender), data.sender.status.name, messagedata_to_msnp(data))
 	
 	def on_close(self) -> None:
 		self.ctrl.close()
@@ -462,9 +482,8 @@ def messagedata_from_msnp(sender: User, sender_pop_id: Optional[str], ack: str,
 		else:
 			pass
 	else:
-		#type = MessageType.Chat
-		#text = "CrossTalk System Message: This message cannot be viewed on the client you're currently using."
-		pass
+		type = MessageType.Chat
+		text = "CrossTalk System Message: This message cannot be viewed on the client you're currently using."
 
 	message = MessageData(sender = sender, sender_pop_id = sender_pop_id, type = type, text = text)
 	message.front_cache['msnp'] = data

front/oscar/ctrl.py

@@ -2,14 +2,16 @@ import asyncio
 import random
 import struct
 
+from core import error
 from core.backend import Backend, BackendSession, Chat, ChatSession
 from core.client import Client
-from core.models import LoginOption
+from core.models import LoginOption, ContactList
 from itertools import cycle
 from typing import Optional, Callable, Tuple
 from util.misc import Logger
 
 from .proto.backend import BackendEventHandler, FOODGROUP_VERSIONS, login, LoginError, bos_cookies
+from .proto.chat import ChatEventHandler
 from .proto.snac import OSCARClient, OSCARContext, SNACMessage, foodgroups
 from .proto.tlv import unmarshal_tlvs, find_tlv
 
@@ -78,25 +80,37 @@ class OSCARCtrl:
 
 			if (cookie_tlv := find_tlv(tlvs, 0x0006)) is not None:
 				found = False
+				is_service_connection = False
 
 				for i, d in enumerate(bos_cookies):
 					for cookie in d.items():
 						if cookie[0] == cookie_tlv.data:
-							if hasattr(cookie[1], 'user') and getattr(cookie[1].user, 'uuid', None) is not None:
-								uuid = cookie[1].user.uuid
-							elif getattr(cookie[1], 'uuid', None) is not None:
-								uuid = cookie[1].uuid
-							else:
-								uuid = cookie[1]
-
 							self.logger.info('found BOS cookie')
 
-							self.context.bs = self.backend.login(uuid,
+							if hasattr(cookie[1], 'user') and getattr(cookie[1].user, 'uuid', None) is not None:
+								# cookie maps to an existing BackendSession (placed there by OSERVICE__SERVICE_REQUEST)
+								# reuse it directly so the main session is not booted and its substatus is preserved
+								self.context.bs = cookie[1]
+								self.context.user = cookie[1].user
+								self.bs = cookie[1]
+								is_service_connection = True
+							else:
+								if isinstance(cookie[1], dict):
+									uuid = cookie[1].get('uuid')
+									version = cookie[1].get('version', '?')
+									self.client = Client('aim', version, self.client.via)
+									self.context.client = self.client
+								elif getattr(cookie[1], 'uuid', None) is not None:
+									uuid = cookie[1].uuid
+								else:
+									uuid = cookie[1]
+
+								self.context.bs = self.backend.login(uuid,
 																 self.client,
 																 BackendEventHandler(self),
 																 option=LoginOption.BootOthers)
-							self.context.user = self.context.bs.user
-							self.bs = self.context.bs
+								self.context.user = self.context.bs.user
+								self.bs = self.context.bs
 
 							bos_cookies.pop(i)
 							found = True
@@ -114,8 +128,12 @@ class OSCARCtrl:
 						msg.write_u16(foodgroup)
 
 					self.send_snac(msg)
-					self.bs.front_data['oscar0'] = True
-					self.bs.front_data['oscar0_chats'] = {}
+
+					if not is_service_connection:
+						# only initialize these keys on the primary BOS connection
+						# so other service connections don't wipe the chat state built up so far
+						self.bs.front_data['oscar0'] = True
+						self.bs.front_data['oscar0_chats'] = {}
 				else:
 					self.logger.info('invalid BOS cookie given')
 

front/oscar/entry.py

@@ -4,7 +4,6 @@ import settings
 
 from core.backend import Backend
 from typing import Optional, Callable
-from threading import Thread
 from util.misc import Logger, ProtocolRunner
 
 from .ctrl import OSCARCtrl
@@ -22,7 +21,6 @@ class ListenerOSCAR(asyncio.Protocol):
 	transport: Optional[asyncio.WriteTransport]
 
 	buffer: bytes = b''
-	data_thread: Thread = None
 
 	def __init__(self,
 				 logger_prefix: str,
@@ -51,13 +49,14 @@ class ListenerOSCAR(asyncio.Protocol):
 			return
 
 		self.buffer += packet
-
-		if self.data_thread is None or not self.data_thread.is_alive():
-			self.data_thread = Thread(target=self.parse_buffer)
-			self.data_thread.start()
+		self.parse_buffer()
 
 	def parse_buffer(self) -> None:
 		while True:
+			if self.controller.closed:
+				self.buffer = b''
+				break
+
 			if self.buffer[0] != 0x2A:
 				break
 
@@ -106,4 +105,4 @@ class ListenerOSCAR(asyncio.Protocol):
 
 	def _on_close(self) -> None:
 		if self.transport is not None:
-			self.transport.close()
+			self.transport.close()

front/oscar/foodgroups/__init__.py

@@ -6,5 +6,6 @@ __all__ = [
 	'bos',
 	'stats',
 	'feedbag',
+	'popup',
 	'bucp'
 ]

front/oscar/foodgroups/bucp.py

@@ -1,7 +1,9 @@
+from core.client import Client
 from util.misc import Logger
 from util.hash import gen_salt
 
 from ..proto.backend import login, LoginError
+from ..proto.buffer import Buffer
 from ..proto.snac import OSCARClient, OSCARContext, SNACMessage, Foodgroup, Subgroup
 from ..proto.tlv import unmarshal_tlvs, find_tlv
 
@@ -40,29 +42,47 @@ class BUCPFoodgroup:
 		tlvs = unmarshal_tlvs(message.data)
 
 		screen_name_tlv = find_tlv(tlvs, 0x0001)
-		hashed_pw_tlv = find_tlv(tlvs, 0x0025)
+		hashed_pw_tlv   = find_tlv(tlvs, 0x0025)
 
 		screen_name = screen_name_tlv.data.decode()
 
+		major_tlv = find_tlv(tlvs, 0x0017)
+		minor_tlv = find_tlv(tlvs, 0x0018)
+		build_tlv = find_tlv(tlvs, 0x001A)
+
+		major = Buffer(major_tlv.data).read_u16() if major_tlv else 0
+		minor = Buffer(minor_tlv.data).read_u16() if minor_tlv else 0
+		build = Buffer(build_tlv.data).read_u16() if build_tlv else 0
+
+		version_str = '{}.{}.{}'.format(major, minor, build)
+		context.client = Client('aim', version_str, context.client.via)
+
 		self.logger.info('Screen Name (client-given):', screen_name)
+		self.logger.info('Client version:', version_str)
 		self.logger.info('Password (hashed):', hashed_pw_tlv.data)
 
 		response_msg = SNACMessage(0x0017, 0x0003)
 
 		error_code = None
+		uuid = None
 
 		if (uuid := context.backend.util_get_uuid_from_username(screen_name)) is None:
 			error_code = LoginError.UnregisteredScreenname
-
 			self.logger.info('Unregistered screenname')
 		else:
-			password = context.backend.user_service.aim_get_md5_password(screen_name)
+			client_hash = hashed_pw_tlv.data
 
-			if password != hashed_pw_tlv.data:
+			pw_bucp1 = context.backend.user_service.aim_get_md5_password(screen_name)
+			pw_bucp2 = context.backend.user_service.aim_get_md5_password_bucp2(screen_name)
+
+			auth_ok = (pw_bucp1 is not None and pw_bucp1 == client_hash) or (pw_bucp2 is not None and pw_bucp2 == client_hash)
+
+			if not auth_ok:
 				error_code = LoginError.IncorrectPassword
-
 				self.logger.info('Incorrect password')
+			else:
+				self.logger.info('Authenticated via', 'AIM5.x' if (pw_bucp2 == client_hash) else 'AIM3.5-4.8')
 
 		response_msg.write_bytes(login(self.logger, context, tlvs, uuid, error_code))
 
-		client.send_snac(response_msg)
+		client.send_snac(response_msg)

front/oscar/foodgroups/feedbag.py

@@ -2,6 +2,7 @@ import time
 import struct
 
 from array import array
+from core import error
 from core.models import ContactList, Substatus
 from dataclasses import dataclass
 from enum import IntEnum
@@ -16,10 +17,12 @@ from ..proto.buffer import Buffer
 
 # I only put the classes I need in this IntEnum
 #
-# For a complete list of feedbag classes, see https://wiki.nina.chat/wiki/Protocols/OSCAR/Foodgroups/FEEDBAG/Items#Class:_FEEDBAG_CLASS_IDS
+# For a complete list of feedbag classes, see https://ninawiki.preloading.dev/wiki/Protocols/OSCAR/Foodgroups/FEEDBAG/Items#Class:_FEEDBAG_CLASS_IDS
 class FeedbagClass(IntEnum):
 	Buddy = 0x0000,
 	Group = 0x0001,
+	Permit = 0x0002,
+	Deny = 0x0003,
 	PdInfo = 0x0004,
 	BuddyPrefs = 0x0005
 
@@ -34,6 +37,13 @@ class FeedbagAttributes(IntEnum):
 	PdMask = 0x00CB,
 	BuddyPrefs = 0x00C9
 
+class FeedbagStatus(IntEnum):
+	Success = 0x0000,
+	NotFound = 0x0002,
+	AlreadyExists = 0x0003,
+	BadRequest = 0x000A,
+	OverLimit = 0x000C,
+	BadLoginID = 0x0010,
 
 @dataclass
 class FeedbagItem:
@@ -71,12 +81,12 @@ def get_items(context: OSCARContext) -> array[FeedbagItem]:
 	})
 
 	group_feedbag = []
-	group_order = b''
+	group_order = b""
 
 	contact_feedbag = []
 
 	for id, group in detail._groups_by_id.items():
-		order = b''
+		order = b""
 
 		for contact in contacts:
 			user = contact.head
@@ -84,47 +94,47 @@ def get_items(context: OSCARContext) -> array[FeedbagItem]:
 			for grp in contact._groups.copy():
 				if grp.id == id:
 					contact_feedbag.append(FeedbagItem(user.username, int(group.id), user.id, FeedbagClass.Buddy))
-					order += struct.pack('>H', user.id)
+					order += struct.pack(">H", user.id)
 
 		group_feedbag.append(FeedbagItem(group.name, int(group.id), 0x0000, FeedbagClass.Group, [
 			TLV(FeedbagAttributes.Order, order)
 		]))
 
-		group_order += struct.pack('>H', int(group.id))
+		group_order += struct.pack(">H", int(group.id))
 
 	# deal with any ungrouped contacts left
 	if ungrouped_contacts := [contact for contact in contacts if not contact._groups]:
 		no_group_gid = len(group_feedbag) + 1
 
-		order = b''
+		order = b""
 
 		for contact in ungrouped_contacts:
 			user = contact.head
 
 			contact_feedbag.append(FeedbagItem(user.username, no_group_gid, user.id, FeedbagClass.Buddy))
 
-			order += struct.pack('>H', user.id)
+			order += struct.pack(">H", user.id)
 
-		group_feedbag.append(FeedbagItem('(No Group)', no_group_gid, 0x0000, FeedbagClass.Group, [
+		group_feedbag.append(FeedbagItem("(No Group)", no_group_gid, 0x0000, FeedbagClass.Group, [
 			TLV(FeedbagAttributes.Order, order)
 		]))
 
 	return [
-		FeedbagItem('', 0x0000, 0x0000, FeedbagClass.Group, [
+		FeedbagItem("", 0x0000, 0x0000, FeedbagClass.Group, [
 			TLV(FeedbagAttributes.Order, group_order)
 		]),
 
-		FeedbagItem('', 0x0000, 0x0E63, FeedbagClass.PdInfo, [
-			TLV(FeedbagAttributes.PdMode,			   struct.pack('>H', 0x0004)),
-			TLV(FeedbagAttributes.WirelessPdMode,	   struct.pack('>B', 0x0001)),
-			TLV(FeedbagAttributes.WirelessIgnoreMode,   struct.pack('>B', 0x0001)),
-			TLV(FeedbagAttributes.FishPdMode,		   struct.pack('>B', 0x0001)),
-			TLV(FeedbagAttributes.FishIgnoreMode,	   struct.pack('>B', 0x0001)),
-			TLV(FeedbagAttributes.PdMask,			   struct.pack('>H', 0xFFFF))
+		FeedbagItem("", 0x0000, 0x0E63, FeedbagClass.PdInfo, [
+			TLV(FeedbagAttributes.PdMode,			struct.pack(">H", 0x0004)),
+			TLV(FeedbagAttributes.WirelessPdMode,	struct.pack(">B", 0x0001)),
+			TLV(FeedbagAttributes.WirelessIgnoreMode, struct.pack(">B", 0x0001)),
+			TLV(FeedbagAttributes.FishPdMode,		struct.pack(">B", 0x0001)),
+			TLV(FeedbagAttributes.FishIgnoreMode,	struct.pack(">B", 0x0001)),
+			TLV(FeedbagAttributes.PdMask,			struct.pack(">H", 0xFFFF))
 		]),
 
-		FeedbagItem('', 0x0000, 0x4B1D, FeedbagClass.BuddyPrefs, [
-			TLV(FeedbagAttributes.BuddyPrefs, struct.pack('>L', 0x00000400))
+		FeedbagItem("", 0x0000, 0x4B1D, FeedbagClass.BuddyPrefs, [
+			TLV(FeedbagAttributes.BuddyPrefs, struct.pack(">L", 0x00000400))
 		]),
 
 		*group_feedbag,
@@ -133,7 +143,7 @@ def get_items(context: OSCARContext) -> array[FeedbagItem]:
 
 
 def marshal_items(items: array[FeedbagItem]) -> bytes:
-	return b''.join([item.marshal() for item in items])
+	return b"".join([item.marshal() for item in items])
 
 
 def unmarshal_items(item_bytes: bytes) -> array[FeedbagItem]:
@@ -159,23 +169,46 @@ def unmarshal_items(item_bytes: bytes) -> array[FeedbagItem]:
 	return items
 
 
+def _send_feedbag_status(client: OSCARClient, request_id: int, statuses: list) -> None:
+	msg = SNACMessage(0x0013, 0x000E, 0x0000, request_id)
+	for code in statuses:
+		msg.write_u16(code)
+	client.send_snac(msg)
+
+
+def _resolve_group_backend_id(context: OSCARContext, client_group_id: int) -> Optional[str]:
+	detail = context.user.detail
+	if detail is None:
+		return None
+	if str(client_group_id) in detail._groups_by_id:
+		return str(client_group_id)
+	mapping = context.bs.front_data.get("oscar0_group_id_map", {})
+	return mapping.get(client_group_id)
+
+def build_buddy_added_notif(adder_username: str) -> SNACMessage:
+	msg = SNACMessage(0x0013, 0x001C)
+	username_bytes = adder_username.encode('ascii')
+	msg.write_u8(len(username_bytes))
+	msg.write_bytes(username_bytes)
+	return msg
+
 @Foodgroup(0x0013)
 class FeedbagFoodgroup:
 	logger: Logger
 
 	@Subgroup(0x0002)
 	def rights_query(self, client: OSCARClient, context: OSCARContext, message: SNACMessage) -> None:
-		self.logger.info('[Client] FEEDBAG__RIGHTS_QUERY')
+		self.logger.info("[Client] FEEDBAG__RIGHTS_QUERY")
 
 		response_msg = SNACMessage(0x0013, 0x0003)
 
-		# https://wiki.nina.chat/wiki/Protocols/OSCAR/SNAC/FEEDBAG_RIGHTS_REPLY#TLV_Class:_FEEDBAG_RIGHTS_REPLY_TAGS
+		# https://ninawiki.preloading.dev/wiki/Protocols/OSCAR/SNAC/FEEDBAG_RIGHTS_REPLY#TLV_Class:_FEEDBAG_RIGHTS_REPLY_TAGS
 		response_msg.write_tlvs([
-			TLV(0x0002, struct.pack('>H', 254)),	# max class attrs
-			TLV(0x0003, struct.pack('>H', 1698)),   # max item attrs
+			TLV(0x0002, struct.pack(">H", 254)),	# max class attrs
+			TLV(0x0003, struct.pack(">H", 1698)),   # max item attrs
 
 			# max items
-			TLV(0x0004, struct.pack(f'>{'H' * 67}',
+			TLV(0x0004, struct.pack(f">{'H' * 67}",
 									1000,   # max num of contacts
 									100,	# max num of groups
 									1000,   # max num of visible contacts
@@ -204,24 +237,24 @@ class FeedbagFoodgroup:
 									500, 1, 8, 10000, 1, 1, 1, 10000, 0, 0, 1, 2000, 0, 60, 24, 10, 1, 0, 0,
 									0, 0, 1, 1, 1, 1, 1000, 1, 1)),
 
-			TLV(0x0005, struct.pack('>H', 100)),	 # max client items
-			TLV(0x0006, struct.pack('>H', 97)),	  # max item name len
-			TLV(0x0007, struct.pack('>H', 2000)),	# max recent buddies
-			TLV(0x0008, struct.pack('>H', 10)),	  # interaction buddies
-			TLV(0x0009, struct.pack('>L', 432000)),  # interaction half life - in 2^(-age/half_life) in seconds
-			TLV(0x000A, struct.pack('>L', 14)),	  # interaction max score
-			TLV(0x000B, struct.pack('>H', 0)),	   # unknown
-			TLV(0x000C, struct.pack('>H', 600)),	 # max buddies per group
-			TLV(0x000D, struct.pack('>H', 200)),	 # max allowed bot buddies
-			TLV(0x000E, struct.pack('>H', 32))	   # max smart groups
+			TLV(0x0005, struct.pack(">H", 100)),	# max client items
+			TLV(0x0006, struct.pack(">H", 97)),	 # max item name len
+			TLV(0x0007, struct.pack(">H", 2000)),   # max recent buddies
+			TLV(0x0008, struct.pack(">H", 10)),	 # interaction buddies
+			TLV(0x0009, struct.pack(">L", 432000)), # interaction half life - in 2^(-age/half_life) in seconds
+			TLV(0x000A, struct.pack(">L", 14)),	 # interaction max score
+			TLV(0x000B, struct.pack(">H", 0)),	  # unknown
+			TLV(0x000C, struct.pack(">H", 600)),	# max buddies per group
+			TLV(0x000D, struct.pack(">H", 200)),	# max allowed bot buddies
+			TLV(0x000E, struct.pack(">H", 32))	  # max smart groups
 		])
 
-		self.logger.info('[Server] FEEDBAG__RIGHTS_REPLY')
+		self.logger.info("[Server] FEEDBAG__RIGHTS_REPLY")
 		client.send_snac(response_msg)
 
 	@Subgroup(0x0004)
 	def query(self, client: OSCARClient, context: OSCARContext, message: SNACMessage) -> None:
-		self.logger.info('[Client] FEEDBAG__QUERY')
+		self.logger.info("[Client] FEEDBAG__QUERY")
 
 		items = get_items(context)
 
@@ -231,12 +264,12 @@ class FeedbagFoodgroup:
 		response_msg.write_bytes(marshal_items(items))  # list of feedbag items
 		response_msg.write_u32(int(time.time()))		# feedbag last change time - TODO: should pull from db
 
-		self.logger.info('[Server] FEEDBAG__REPLY')
+		self.logger.info("[Server] FEEDBAG__REPLY")
 		client.send_snac(response_msg)
 
 	@Subgroup(0x0005)
 	def query_if_modified(self, client: OSCARClient, context: OSCARContext, message: SNACMessage) -> None:
-		self.logger.info('[Client] FEEDBAG__QUERY_IF_MODIFIED (not implemented)')
+		self.logger.info("[Client] FEEDBAG__QUERY_IF_MODIFIED")
 
 		# 66 51 29 47 00 0D
 		#
@@ -245,19 +278,28 @@ class FeedbagFoodgroup:
 		cached_feedbag_timestamp = message.read_u32()
 		cached_feedbag_num_items = message.read_u16()
 
-		self.logger.info('[Client] Cached feedbag timestamp:', cached_feedbag_timestamp)
-		self.logger.info('[Client] Cached feedbag items num:', cached_feedbag_num_items)
+		self.logger.info("[Client] Cached feedbag timestamp:", cached_feedbag_timestamp)
+		self.logger.info("[Client] Cached feedbag items num:", cached_feedbag_num_items)
 
-		# TODO(subpurple): do check
-		self.query(client, context, message)
+		current_items = get_items(context)
+
+		if len(current_items) == cached_feedbag_num_items:
+			response_msg = SNACMessage(0x0013, 0x000F, 0x0000, message.request_id)
+			response_msg.write_u32(cached_feedbag_timestamp)
+			response_msg.write_u16(cached_feedbag_num_items)
+
+			self.logger.info("[Server] FEEDBAG__REPLY_NOT_MODIFIED")
+			client.send_snac(response_msg)
+		else:
+			self.query(client, context, message)
 
 	@Subgroup(0x0007)
 	def use(self, client: OSCARClient, context: OSCARContext, message: SNACMessage) -> None:
-		self.logger.info('[Client] FEEDBAG__USE')
+		self.logger.info("[Client] FEEDBAG__USE")
 
 		# set our status to Online
 		context.bs.me_update({
-			'substatus': Substatus.Online
+			"substatus": Substatus.Online
 		})
 
 		# notify the client if any contacts are online
@@ -275,23 +317,271 @@ class FeedbagFoodgroup:
 	# stubs because I cba to implement more of feedbag rn
 	@Subgroup(0x0011)
 	def start_cluster(self, client: OSCARClient, context: OSCARContext, message: SNACMessage) -> None:
-		self.logger.info('[Client] FEEDBAG__START_CLUSTER (not implemented)')
+		self.logger.info("[Client] FEEDBAG__START_CLUSTER (not implemented)")
 
 	@Subgroup(0x0012)
 	def end_cluster(self, client: OSCARClient, context: OSCARContext, message: SNACMessage) -> None:
-		self.logger.info('[Client] FEEDBAG__END_CLUSTER (not implemented)')
+		self.logger.info("[Client] FEEDBAG__END_CLUSTER (not implemented)")
 
 	@Subgroup(0x0008)
 	def insert_item(self, client: OSCARClient, context: OSCARContext, message: SNACMessage) -> None:
-		self.logger.info('[Client] FEEDBAG__INSERT_ITEM (not implemented)')
-		self.logger.info('[Client]', unmarshal_items(message.data))
+		self.logger.info("[Client] FEEDBAG__INSERT_ITEM")
+
+		bs = context.bs
+		user = context.user
+		detail = user.detail
+		items = unmarshal_items(message.data)
+		statuses = []
+
+		for item in items:
+			if item.class_id == FeedbagClass.Group:
+				if item.group_id == 0 and item.item_id == 0:
+					statuses.append(FeedbagStatus.Success)
+					continue
+
+				existing_names = {g.name.lower() for g in detail._groups_by_id.values()}
+				if item.name.lower() in existing_names:
+					statuses.append(FeedbagStatus.AlreadyExists)
+					continue
+
+				try:
+					group = bs.me_group_add(item.name)
+					mapping = bs.front_data.setdefault("oscar0_group_id_map", {})
+					mapping[item.group_id] = group.id
+					self.logger.info("[Client] Added group:", item.name)
+					statuses.append(FeedbagStatus.Success)
+				except Exception:
+					statuses.append(FeedbagStatus.BadRequest)
+
+			elif item.class_id == FeedbagClass.Buddy:
+				if not item.name:
+					statuses.append(FeedbagStatus.BadLoginID)
+					continue
+
+				contact_uuid = context.backend.util_get_uuid_from_username(item.name)
+				if contact_uuid is None:
+					self.logger.info("[Client] Unknown buddy:", item.name)
+					statuses.append(FeedbagStatus.BadLoginID)
+					continue
+
+				contact = detail.contacts.get(contact_uuid)
+				if contact is not None and contact.lists & ContactList.FL:
+					statuses.append(FeedbagStatus.AlreadyExists)
+					continue
+
+				try:
+					bs.me_contact_add(contact_uuid, ContactList.FL | ContactList.AL)
+
+					if item.group_id != 0:
+						backend_group_id = _resolve_group_backend_id(context, item.group_id)
+						if backend_group_id is not None:
+							bs.me_group_contact_add(backend_group_id, contact_uuid)
+
+					self.logger.info("[Client] Added buddy:", item.name)
+					statuses.append(FeedbagStatus.Success)
+				except error.ContactListIsFull:
+					statuses.append(FeedbagStatus.OverLimit)
+				except Exception:
+					statuses.append(FeedbagStatus.BadRequest)
+
+			elif item.class_id == FeedbagClass.Deny:
+				if not item.name:
+					statuses.append(FeedbagStatus.BadLoginID)
+					continue
+
+				contact_uuid = context.backend.util_get_uuid_from_username(item.name)
+				if contact_uuid is None:
+					statuses.append(FeedbagStatus.BadLoginID)
+					continue
+
+				contact = detail.contacts.get(contact_uuid)
+				if contact is not None and contact.lists & ContactList.BL:
+					statuses.append(FeedbagStatus.AlreadyExists)
+					continue
+
+				try:
+					bs.me_contact_add(contact_uuid, ContactList.BL)
+					self.logger.info("[Client] Added deny entry:", item.name)
+					statuses.append(FeedbagStatus.Success)
+				except Exception:
+					statuses.append(FeedbagStatus.BadRequest)
+
+			elif item.class_id == FeedbagClass.Permit:
+				if not item.name:
+					statuses.append(FeedbagStatus.BadLoginID)
+					continue
+
+				contact_uuid = context.backend.util_get_uuid_from_username(item.name)
+				if contact_uuid is None:
+					statuses.append(FeedbagStatus.BadLoginID)
+					continue
+
+				contact = detail.contacts.get(contact_uuid)
+				if contact is not None and contact.lists & ContactList.AL:
+					statuses.append(FeedbagStatus.AlreadyExists)
+					continue
+
+				try:
+					bs.me_contact_add(contact_uuid, ContactList.AL)
+					self.logger.info("[Client] Added permit entry:", item.name)
+					statuses.append(FeedbagStatus.Success)
+				except Exception:
+					statuses.append(FeedbagStatus.BadRequest)
+
+			else:
+				statuses.append(FeedbagStatus.Success)
+
+		self.logger.info("[Server] FEEDBAG__STATUS")
+		_send_feedbag_status(client, message.request_id, statuses)
 
 	@Subgroup(0x0009)
 	def update_item(self, client: OSCARClient, context: OSCARContext, message: SNACMessage) -> None:
-		self.logger.info('[Client] FEEDBAG__UPDATE_ITEM (not implemented)')
-		self.logger.info('[Client]', unmarshal_items(message.data))
+		self.logger.info("[Client] FEEDBAG__UPDATE_ITEM")
+
+		bs = context.bs
+		user = context.user
+		detail = user.detail
+		items = unmarshal_items(message.data)
+		statuses = []
+
+		for item in items:
+			if item.class_id == FeedbagClass.Group:
+				if item.group_id == 0 and item.item_id == 0:
+					statuses.append(FeedbagStatus.Success)
+					continue
+
+				backend_group_id = _resolve_group_backend_id(context, item.group_id)
+				if backend_group_id is None:
+					statuses.append(FeedbagStatus.NotFound)
+					continue
+
+				group = detail._groups_by_id.get(backend_group_id)
+				if group is None:
+					statuses.append(FeedbagStatus.NotFound)
+					continue
+
+				if item.name and item.name != group.name:
+					try:
+						bs.me_group_edit(backend_group_id, new_name=item.name)
+						self.logger.info("[Client] Renamed group to:", item.name)
+						statuses.append(FeedbagStatus.Success)
+					except Exception:
+						statuses.append(FeedbagStatus.BadRequest)
+				else:
+					statuses.append(FeedbagStatus.Success)
+
+			else:
+				statuses.append(FeedbagStatus.Success)
+
+		self.logger.info("[Server] FEEDBAG__STATUS")
+		_send_feedbag_status(client, message.request_id, statuses)
 
 	@Subgroup(0x000A)
 	def delete_item(self, client: OSCARClient, context: OSCARContext, message: SNACMessage) -> None:
-		self.logger.info('[Client] FEEDBAG__DELETE_ITEM (not implemented)')
-		self.logger.info('[Client]', unmarshal_items(message.data))
+		self.logger.info("[Client] FEEDBAG__DELETE_ITEM")
+
+		bs = context.bs
+		user = context.user
+		detail = user.detail
+		items = unmarshal_items(message.data)
+		statuses = []
+
+		for item in items:
+			if item.class_id == FeedbagClass.Group:
+				if item.group_id == 0 and item.item_id == 0:
+					statuses.append(FeedbagStatus.BadRequest)
+					continue
+
+				backend_group_id = _resolve_group_backend_id(context, item.group_id)
+				if backend_group_id is None:
+					statuses.append(FeedbagStatus.NotFound)
+					continue
+
+				try:
+					bs.me_group_remove(backend_group_id)
+					mapping = bs.front_data.get("oscar0_group_id_map", {})
+					for k in [k for k, v in mapping.items() if v == backend_group_id]:
+						del mapping[k]
+					self.logger.info("[Client] Removed group:", item.name)
+					statuses.append(FeedbagStatus.Success)
+				except Exception:
+					statuses.append(FeedbagStatus.NotFound)
+
+			elif item.class_id == FeedbagClass.Buddy:
+				if not item.name:
+					statuses.append(FeedbagStatus.BadLoginID)
+					continue
+
+				contact_uuid = context.backend.util_get_uuid_from_username(item.name)
+				if contact_uuid is None:
+					statuses.append(FeedbagStatus.BadLoginID)
+					continue
+
+				contact = detail.contacts.get(contact_uuid)
+				if contact is None or not (contact.lists & ContactList.FL):
+					statuses.append(FeedbagStatus.NotFound)
+					continue
+
+				try:
+					if item.group_id != 0:
+						backend_group_id = _resolve_group_backend_id(context, item.group_id)
+						if backend_group_id is not None:
+							bs.me_group_contact_remove(backend_group_id, contact_uuid)
+
+					if not contact._groups:
+						bs.me_contact_remove(contact_uuid, ContactList.FL)
+
+					self.logger.info("[Client] Removed buddy:", item.name)
+					statuses.append(FeedbagStatus.Success)
+				except Exception:
+					statuses.append(FeedbagStatus.BadRequest)
+
+			elif item.class_id == FeedbagClass.Deny:
+				if not item.name:
+					statuses.append(FeedbagStatus.BadLoginID)
+					continue
+
+				contact_uuid = context.backend.util_get_uuid_from_username(item.name)
+				if contact_uuid is None:
+					statuses.append(FeedbagStatus.BadLoginID)
+					continue
+
+				contact = detail.contacts.get(contact_uuid)
+				if contact is None or not (contact.lists & ContactList.BL):
+					statuses.append(FeedbagStatus.NotFound)
+					continue
+
+				try:
+					bs.me_contact_remove(contact_uuid, ContactList.BL)
+					self.logger.info("[Client] Removed deny entry:", item.name)
+					statuses.append(FeedbagStatus.Success)
+				except Exception:
+					statuses.append(FeedbagStatus.BadRequest)
+
+			elif item.class_id == FeedbagClass.Permit:
+				if not item.name:
+					statuses.append(FeedbagStatus.BadLoginID)
+					continue
+
+				contact_uuid = context.backend.util_get_uuid_from_username(item.name)
+				if contact_uuid is None:
+					statuses.append(FeedbagStatus.BadLoginID)
+					continue
+
+				contact = detail.contacts.get(contact_uuid)
+				if contact is None or not (contact.lists & ContactList.AL):
+					statuses.append(FeedbagStatus.NotFound)
+					continue
+
+				try:
+					bs.me_contact_remove(contact_uuid, ContactList.AL)
+					self.logger.info("[Client] Removed permit entry:", item.name)
+					statuses.append(FeedbagStatus.Success)
+				except Exception:
+					statuses.append(FeedbagStatus.BadRequest)
+
+			else:
+				statuses.append(FeedbagStatus.Success)
+
+		self.logger.info("[Server] FEEDBAG__STATUS")
+		_send_feedbag_status(client, message.request_id, statuses)

front/oscar/foodgroups/icbm.py

@@ -1,4 +1,6 @@
 import time, struct
+import re
+from html import unescape
 
 from enum import IntEnum
 
@@ -6,14 +8,21 @@ from util.misc import Logger
 from core.models import MessageData, MessageType, User
 
 from ..proto.snac import OSCARClient, OSCARContext, SNACMessage, Foodgroup, Subgroup
-from ..proto.tlv import unmarshal_tlvs, marshal_tlvs, TLV
+from ..proto.tlv import find_tlv, unmarshal_tlvs, marshal_tlvs, TLV
+
 
 class ICBMChannel(IntEnum):
-	AOLIM = 0x0001,
-	Rendezvous = 0x0002,
-	Mime = 0x0003,
-	ICQ = 0x0004,
-	CoBrowser = 0x0005
+	AOLIM	   = 0x0001
+	Rendezvous  = 0x0002
+	Mime		= 0x0003
+	ICQ		 = 0x0004
+	CoBrowser   = 0x0005
+
+class ICBMEvent(IntEnum):
+	Finished = 0x0000 
+	Typed	= 0x0001
+	Typing   = 0x0002
+	Closed   = 0x000F
 
 @Foodgroup(0x0004)
 class ICBMFoodgroup:
@@ -37,8 +46,8 @@ class ICBMFoodgroup:
 		# 0010   00 05 00 00 00 03 02 00 03 84 03 e7 00 00 03 e8   ................
 		#
 		# The response data are not TLVs and are instead WORD/DWORDs so I cannot fit
-		# the names below the hex data.  See https://wiki.nina.chat/wiki/Protocols/OSCAR/SNAC/ICBM_PARAMETER_REPLY and
-		# https://wiki.nina.chat/wiki/Protocols/OSCAR/SNAC/ICBM_ADD_PARAMETERS for more information.
+		# the names below the hex data.  See https://ninawiki.preloading.dev/wiki/Protocols/OSCAR/SNAC/ICBM_PARAMETER_REPLY and
+		# https://ninawiki.preloading.dev/wiki/Protocols/OSCAR/SNAC/ICBM_ADD_PARAMETERS for more information.
 		response_msg.write_u16(5)		 # maxSlots
 		response_msg.write_u32(0x00003)   # icbmFlags (default)
 		response_msg.write_u16(512)	   # maxIncomingICBMLen
@@ -51,93 +60,159 @@ class ICBMFoodgroup:
 
 	@Subgroup(0x0006)
 	def channel_msg_tohost(self, client: OSCARClient, context: OSCARContext, message: SNACMessage) -> None:
-		# Client packet:
-		# ===
-		# 0000  44 42 38 35 30 35 00 00 00 01 04 74 65 73 74 00  DB8505.....test.
-		# 0010  02 00 3F 05 01 00 03 01 01 02 01 01 00 34 00 00  ..?..........4..
-		# 0020  00 00 3C 48 54 4D 4C 3E 3C 42 4F 44 59 20 42 47  ..<HTML><BODY BG
-		# 0030  43 4F 4C 4F 52 3D 22 23 66 66 66 66 66 66 22 3E  COLOR="#ffffff">
-		# 0040  74 65 73 74 3C 2F 42 4F 44 59 3E 3C 2F 48 54 4D  test</BODY></HTM
-		# 0050  4C 3E 00 03 00 00								L>....
-		#
-		# [OSCAR] (ID: c245) [Client] ICBM__CHANNEL_MSG_TOHOST
-		# [OSCAR] (ID: c245) [Client] Cookie: b'E67FA1\x00\x00'
-		# [OSCAR] (ID: c245) [Client] Channel: 0x1
-		# [OSCAR] (ID: c245) [Client] Message reciever: test
-		# [OSCAR] (ID: c245) [Server] TLV 2 - b'\x05\x01\x00\x03\x01\x01\x02\x01\x01\x006\x00\x00\x00\x00<HTML><BODY BGCOLOR="#ffffff">123456</BODY></HTML>'
 		cookie = message.read_bytes(8)
 		channel = message.read_u16()
-		reciever = message.read_string_u8()
+		receiver = message.read_string_u8()
 		tlvs = unmarshal_tlvs(message.data)
 
 		self.logger.info('[Client] ICBM__CHANNEL_MSG_TOHOST')
 		self.logger.info('[Client] Cookie:', cookie)
 		self.logger.info('[Client] Channel:', hex(channel))
-		self.logger.info('[Client] Message reciever:', reciever)
+		self.logger.info('[Client] Message receiver:', receiver)
 
 		for tlv in tlvs:
 			self.logger.info('[Client] TLV', hex(tlv.type), '-', tlv.data)
 
-def messagedata_to_icbm(cookie: bytes, data: MessageData, user: User):
-	# THIS SHIT DOESN'T WORK. WHY??? HAS I EVER!?!?!?
-	if 'icbm' not in data.front_cache:
-		type = data.type
-		text = data.text
-		sender = data.sender
-		if type == MessageType.Chat:
-			msg = SNACMessage(0x0004, 0x0007)
-			msg.write_bytes(cookie)
-			msg.write_u16(ICBMChannel.AOLIM)
+		if channel != ICBMChannel.AOLIM:
+			self.logger.info('[Client] Unsupported channel:', hex(channel))
+			return
 
-			msg.write_string_u8(user.username)
-			msg.write_u16(0)
-			# we should not hardcode these
-			capabilities = [
-				"{09461345-4C7F-11D1-8222-444553540000}",   # Direct ICBM
-				"{094601FF-4C7F-11D1-8222-444553540000}",   # Smart caps
-				"{748F2420-6287-11D1-8222-444553540000}",   # Chat
-				"{09461343-4C7F-11D1-8222-444553540000}",   # File transfer
-				"{09461341-4C7F-11D1-8222-444553540000}",   # Voice chat
-				"{09460104-4C7F-11D1-8222-444553540000}",   # RTC audio
-				"{09460105-4C7F-11D1-8222-444553540000}",   # Unknown
-				"{09460102-4C7F-11D1-8222-444553540000}",   # Camera
-				"{09460103-4C7F-11D1-8222-444553540000}",   # Microphone
-				"{09460101-4C7F-11D1-8222-444553540000}",   # RTC video
-				"{0946134A-4C7F-11D1-8222-444553540000}",   # Games
-				"{09461346-4C7F-11D1-8222-444553540000}"	# BART
-			]
-			capabilities_bytes = b''
+		im_data_tlv = find_tlv(tlvs, 0x0002)
+		if im_data_tlv is None:
+			self.logger.info('[Client] Missing IM data TLV')
+			return
 
-			for capability in capabilities:
-				capabilities_bytes += bytes.fromhex(capability
-													.lstrip('{')
-													.rstrip('}')
-													.replace('-', ''))
+		im_tlvs = unmarshal_tlvs(im_data_tlv.data)
+		im_text_tlv = find_tlv(im_tlvs, 0x0101)
+		if im_text_tlv is None:
+			self.logger.info('[Client] Missing IM text TLV inside IM data')
+			return
 
-			date_created_unix = int(time.mktime(user.date_created.timetuple()))
-			date_login_unix = int(time.mktime(user.date_login.timetuple())) if user.date_login else 0
+		text = im_text_tlv.data[4:].decode('ascii', errors='replace')
+		self.logger.info('[Client] Message text:', text)
 
-			# usually would include 0x001D (BART info), but CrossTalk doesn't support that yet
-			pre = marshal_tlvs([
-				TLV(0x3000, struct.pack('>L', 0x6719674C)),	  # Unknown
-				TLV(0x000D, capabilities_bytes),					# Capability info
-				TLV(0x0001, struct.pack('>H', 0x0008)),		  # User class (bitfield)
-				TLV(0x0003, struct.pack('>L', date_login_unix)),	# Account signon time (unix time_t)
-				TLV(0x000F, struct.pack('>L', 0)),				# Session length
-				TLV(0x0005, struct.pack('>L', date_created_unix)),  # Account creation time (unix time_t)
-			])
+		receiver_uuid = context.backend.util_get_uuid_from_username(receiver)
+		if receiver_uuid is None:
+			self.logger.info('[Client] Unknown receiver:', receiver)
+			return
 
+		try:
+			cs, evt = client.ctrl._get_private_chat_with(receiver_uuid)
+		except Exception as ex:
+			self.logger.info('[Client] Could not create private chat:', str(ex))
+			return
 
-			im_text = data.text.encode('ascii')
-			im_tag = 0x0101
-			im_len = 2 + 2 + len(im_text)
-			imdata_bytes = struct.pack('>HHHH', im_tag, im_len, 0, 0) + im_text
+		ack = SNACMessage(0x0004, 0x000C)
+		ack.request_id = message.request_id
+		ack.write_bytes(cookie)
+		ack.write_u16(channel)
+		ack.write_string_u8(receiver)
+		self.logger.info('[Server] ICBM__HOST_ACK')
+		client.send_snac(ack)
 
-			msg.write_tlv_block([
-				*unmarshal_tlvs(pre),
-				TLV(0x0501, struct.pack('>L', 1)),
-				TLV(0x0101, imdata_bytes)
-			])
+		md = MessageData(sender=context.user, type=MessageType.Chat, text=strip_html_tags(text))
+		md.front_cache['icbm_cookie'] = cookie
+		md.front_cache['icbm_html'] = text
+		evt._send_when_user_joins(receiver_uuid, md)
 
-			return msg
+	@Subgroup(0x0014)
+	def client_event(self, client: OSCARClient, context: OSCARContext, message: SNACMessage) -> None:
+		cookie   = message.read_bytes(8)
+		channel  = message.read_u16()
+		receiver = message.read_string_u8()
+		event	= message.read_u16()
 
+		self.logger.info('[Client] ICBM__CLIENT_EVENT')
+		self.logger.info('[Client] Receiver:', receiver, '| Event:', hex(event))
+
+		if channel != ICBMChannel.AOLIM:
+			self.logger.info('[Client] Unsupported channel:', hex(channel))
+			return
+
+		receiver_uuid = context.backend.util_get_uuid_from_username(receiver)
+		if receiver_uuid is None:
+			self.logger.info('[Client] Unknown receiver:', receiver)
+			return
+
+		# map OSCAR events to respective MessageTypes
+		if event == ICBMEvent.Typing:
+			msg_type = MessageType.Typing
+		elif event in (ICBMEvent.Finished, ICBMEvent.Typed, ICBMEvent.Closed):
+			msg_type = MessageType.TypingDone
+		else:
+			self.logger.info('[Client] Unknown event type:', hex(event))
+			return
+
+		try:
+			cs, evt = client.ctrl._get_private_chat_with(receiver_uuid)
+		except Exception as ex:
+			self.logger.info('[Client] Could not get private chat:', str(ex))
+			return
+
+		md = MessageData(sender=context.user, type=msg_type, text='')
+		md.front_cache['icbm_cookie'] = cookie
+
+		if evt._user_in_chat(receiver_uuid):
+			cs.send_message_to_user(receiver_uuid, md)
+
+def strip_html_tags(text: str) -> str:
+	return unescape(re.compile(r'<[^>]+>').sub('', text))
+
+def messagedata_to_icbm(cookie: bytes, data: MessageData, user: User) -> SNACMessage:
+	#   cookie		 - 8 bytes, chat sess cookie
+	#   channel		- u16, 0x0001 for plain text IM
+	#   sender		 - string08 (u8 length prefix), sender's screen name
+	#   warning level  - u16, always 0
+	#   userinfo TLVs  - write_tlv_block (count-prefixed) containing:
+	#					  TLV(0x0001) user class
+	#					  TLV(0x0003) account signon time (unix time_t)
+	#					  TLV(0x000F) session length in seconds
+	#					  TLV(0x0005) account creation time (unix time_t)
+	#   TLV(0x0002)	- IM data block containing nested TLVs:
+	#					  TLV(0x0501) capabilities (1 byte = 0x01 for plain text)
+	#					  TLV(0x0101) text: u16 encoding + u16 language + bytes
+	sender = data.sender
+	text = data.front_cache.get('icbm_html', data.text or '')
+
+	date_created_unix = int(time.mktime(sender.date_created.timetuple()))
+	date_login_unix = int(time.mktime(sender.date_login.timetuple())) if sender.date_login else 0
+
+	msg = SNACMessage(0x0004, 0x0007)
+	msg.write_bytes(cookie)
+	msg.write_u16(ICBMChannel.AOLIM)
+	msg.write_string_u8(sender.username)
+	msg.write_u16(0)  # warning level
+
+	# count-prefixed userinfo TLV block
+	# usually would include TLV(0x001D) (BART info), but we don't support that yet
+	msg.write_tlv_block([
+		TLV(0x0001, struct.pack('>H', 0x0008)),			# user class
+		TLV(0x0003, struct.pack('>L', date_login_unix)),   # account signon time
+		TLV(0x000F, struct.pack('>L', 0)),				 # session length
+		TLV(0x0005, struct.pack('>L', date_created_unix)), # account creation time
+	])
+
+	im_text = (text or '').encode('ascii', errors='replace')
+
+	# IM data: TLV(0x0002) wrapping nested capability + text TLVs
+	im_data = marshal_tlvs([
+		TLV(0x0501, b'\x01'),							 # capabilities: plain text (1 byte)
+		TLV(0x0101, struct.pack('>HH', 0, 0) + im_text), # encoding=ASCII, language=0, text
+	])
+	msg.write_tlv(TLV(0x0002, im_data))
+
+	return msg
+
+def messagedata_to_icbm_event(data: MessageData, receiver: User) -> SNACMessage:
+
+	if data.type is MessageType.Typing:
+		event = ICBMEvent.Typing
+	else:
+		event = ICBMEvent.Finished
+
+	msg = SNACMessage(0x0004, 0x0014)
+	msg.write_bytes(b'\x00' * 8)
+	msg.write_u16(ICBMChannel.AOLIM)
+	msg.write_string_u8(data.sender.username)
+	msg.write_u16(event)
+	return msg

front/oscar/foodgroups/locate.py

@@ -61,7 +61,7 @@ class LocateFoodgroup:
 		#		^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 		#
 		# I have no idea what any of the values in the SNAC body mean, and the lack of documentation on NINA's wiki
-		# don't help either: https://wiki.nina.chat/wiki/Protocols/OSCAR/SNAC/LOCATE_GET_DIR_REPLY
+		# don't help either: https://ninawiki.preloading.dev/wiki/Protocols/OSCAR/SNAC/LOCATE_GET_DIR_REPLY
 		response_msg = SNACMessage(0x0002, 0x000C)
 		response_msg.write_u16(0x0001)
 		response_msg.write_u16(0x0001)

front/oscar/proto/backend.py

@@ -12,6 +12,7 @@ from urllib.parse import quote
 from typing import Optional, Any, Dict
 
 from ..foodgroups.buddy import build_presence_notif
+from ..foodgroups.feedbag import build_buddy_added_notif
 from ..foodgroups.popup import popup_display
 from ..proto.snac import OSCARContext
 from ..proto.tlv import TLV, marshal_tlvs, find_tlv
@@ -39,7 +40,9 @@ FOODGROUP_VERSIONS: {int, int} = {
 ERROR_URLS: {int, str} = {
 	0x0001: 'http://www.aim.aol.com/errors/UNREGISTERED_SCREENNAME.html',   # Unregistered screen name
 	0x0005: 'http://www.aim.aol.com/errors/MISMATCH_PASSWD.html',		   # Incorrect password
+	0x0009: '', # Not a tester (inaccurate)
 	0x0011: 'http://www.aim.aol.com/errors/SUSPENDED.html',				 # Suspended
+	0x0006: '' # Unverified E-mail (inaccurate)
 }
 
 PW_CHANGE_URL_FORMAT = 'http://aim.aol.com/redirects/password/change_password.adp?ScreenName={}&ccode={}&lang={}'
@@ -122,7 +125,8 @@ Now is a good time to wrap up any conversations.""".format(args[1])
 	def on_added_me(self, user: User, *, adder_id: Optional[str] = None,
 					message: Optional[TextWithData] = None) -> None:
 		self.ctrl.logger.info('on_added_me')
-		pass
+
+		self.ctrl.send_snac(build_buddy_added_notif(user.username))
 
 	def on_removed_me(self, user: User) -> None:
 		self.ctrl.logger.info('on_removed_me')
@@ -168,7 +172,8 @@ class LoginError(IntEnum):
 	UnregisteredScreenname = 0x0001,
 	IncorrectPassword = 0x0005,
 	NotATester = 0x0009,
-	Suspended = 0x0011
+	Suspended = 0x0011,
+	EmailNotVerified = 0x0006
 
 
 def login(logger: Logger,
@@ -189,11 +194,12 @@ def login(logger: Logger,
 		error_code = LoginError.Suspended
 
 		logger.info(screen_name, 'tried to sign on but is suspended!')
-
-	if error_code is None and not user.is_tester:
-		error_code = LoginError.NotATester
-
-		logger.info(screen_name, 'tried to sign on but is not a tester!')
+	
+	# and check if the user has verified their e-mail address
+	#if error_code is None and not user.verified_to_login:
+	#	error_code = LoginError.EmailNotVerified
+	#
+	#	logger.info(screen_name, 'tried to sign on but has not verified their e-mail address!')
 		
 	if error_code is None:
 		logger.info(screen_name, 'signed on successfully!')
@@ -201,7 +207,7 @@ def login(logger: Logger,
 		# generate BOS cookie and add it to array
 		bos_cookie = os.urandom(256)
 		bos_cookies.append({
-			bos_cookie: uuid
+			bos_cookie: {'uuid': uuid, 'version': context.client.version}
 		})
 
 		# get user email
@@ -236,4 +242,4 @@ def login(logger: Logger,
 		TLV(0x0001, screen_name),						   # Screen name
 		TLV(0x0008, struct.pack('>H', error_code)),		 # Error code
 		TLV(0x0004, ERROR_URLS[error_code]),				# Error URL
-	])
+	])

front/oscar/proto/chat.py

@@ -1,61 +1,80 @@
 import asyncio
 import secrets
-
-from typing import Any, Optional
+from typing import Any, Optional, List
 from core import event
 from core.backend import BackendSession, ChatSession, Chat
 from core.models import User, Substatus, MessageData, MessageType
-
 from ..proto.snac import SNACMessage
 from ..proto.tlv import TLV
 from ..proto.buffer import Buffer
-from ..foodgroups.icbm import ICBMChannel, messagedata_to_icbm
+from ..foodgroups.icbm import ICBMChannel, messagedata_to_icbm, messagedata_to_icbm_event
 
 class ChatEventHandler(event.ChatEventHandler):
-	__slots__ = ('loop', 'ctrl', 'bs', 'cs', 'cookie')
-
+	__slots__ = ('loop', 'ctrl', 'bs', 'cs', 'cookie', '_pending')
 	loop: asyncio.AbstractEventLoop
 	ctrl: Any
 	bs: BackendSession
 	cs: ChatSession
 	cookie: bytes
+	_pending: List
 
 	def __init__(self, loop: asyncio.AbstractEventLoop, ctrl: Any, bs: BackendSession) -> None:
 		self.loop = loop
 		self.ctrl = ctrl
 		self.bs = bs
 		self.cookie = secrets.token_bytes(8)
+		self._pending = []
 
 	def on_participant_joined(self, cs_other: 'ChatSession', first_pop: bool, initial_join: bool) -> None:
 		self.ctrl.logger.info('on_participant_joined')
-		pass
+		still_pending = []
+		for (target_uuid, data) in self._pending:
+			if cs_other.user.uuid == target_uuid:
+				self.cs.send_message_to_user(target_uuid, data)
+			else:
+				still_pending.append((target_uuid, data))
+		self._pending = still_pending
 
 	def on_participant_left(self, cs_other: 'ChatSession', last_pop: bool) -> None:
 		self.ctrl.logger.info('on_participant_left')
-		pass
 
 	def on_chat_invite_declined(self, chat: 'Chat', invitee: User, *, invitee_id: Optional[str] = None,
 								message: Optional[str] = None, circle: bool = False) -> None:
 		self.ctrl.logger.info('on_chat_invite_declined')
-		pass
 
 	def on_chat_updated(self) -> None:
 		self.ctrl.logger.info('on_chat_updated')
-		pass
 
 	def on_chat_roster_updated(self) -> None:
 		self.ctrl.logger.info('on_chat_roster_updated')
-		pass
 
 	def on_participant_status_updated(self, cs_other: 'ChatSession', first_pop: bool, initial: bool,
 									  old_substatus: Substatus) -> None:
 		self.ctrl.logger.info('on_participant_status_updated')
-		pass
 
 	def on_message(self, data: MessageData) -> None:
+		if data.sender is self.bs.user:
+			if data.front_cache.get('icbm_echo'):
+				original_cookie = data.front_cache.get('icbm_cookie', self.cookie)
+				self.ctrl.send_snac(messagedata_to_icbm(original_cookie, data, self.bs.user))
+			return
+
+		if data.type in (MessageType.Typing, MessageType.TypingDone):
+			self.ctrl.send_snac(messagedata_to_icbm_event(data, self.bs.user))
+			return
 
 		self.ctrl.logger.info('Got a message from', data.sender.username, 'saying:')
 		self.ctrl.logger.info(data.text.encode())
-
 		self.ctrl.send_snac(messagedata_to_icbm(self.cookie, data, self.bs.user))
 
+	def _send_when_user_joins(self, user_uuid: str, data: MessageData) -> None:
+		if self._user_in_chat(user_uuid):
+			self.cs.send_message_to_user(user_uuid, data)
+		else:
+			self._pending.append((user_uuid, data))
+
+	def _user_in_chat(self, user_uuid: str) -> bool:
+		for cs_other in self.cs.chat.get_roster():
+			if cs_other.user.uuid == user_uuid:
+				return True
+		return False

front/ymsg/http.py

@@ -37,6 +37,7 @@ def register(app: web.Application, *, devmode: bool = False) -> None:
 	app.router.add_get('/c/msg/alerts.html', handle_chat_alertad)
 	app.router.add_get('/etc/yahoo-placeholder', handle_placeholder)
 	app.router.add_get('/external/client_ad.php', handle_banneradredir)
+	app.router.add_get('/client_ad.php', handle_banneradredir)
 	
 	# Yahoo!'s redirector to cookie-based services
 	#app.router.add_get('/config/reset_cookies', handle_cookies_redirect)
@@ -51,6 +52,7 @@ def register(app: web.Application, *, devmode: bool = False) -> None:
 
 	# Misc stuff
 	app.router.add_get('/capacity', handle_capacity)
+	app.router.add_get('/nofriends/', handle_nofriends)
 
 async def handle_insider_ycontent(req: web.Request) -> web.Response:
 	backend = req.app['backend']
@@ -326,6 +328,9 @@ async def handle_yahoo_filedl(req: web.Request) -> web.Response:
 async def handle_capacity(req: web.Request) -> web.Response:
 	return web.Response(text="COLO_CAPACITY=1\nCS_IP_ADDRESS={}".format(settings.TARGET_IP))
 
+async def handle_nofriends(req: web.Request) -> web.Response:
+	return render(req, 'ymsg:layout/nofriends.html')
+
 async def handle_gettoken(req: web.Request) -> web.Response:
 	backend = req.app['backend']
 

front/ymsg/misc.py

@@ -32,6 +32,7 @@ class YMSGService(IntEnum):
 	ConfMsg = 0x1D
 	MessageV2 = 0x27
 	AvatarOld = 0xBD
+	Avatar6 = 0xC1
 	Avatar = 0xC7
 	FileTransfer = 0x46
 	VoiceChat = 0x4A
@@ -60,6 +61,7 @@ class YMSGService(IntEnum):
 	ChatPing = 0xA1
 	ChatSession = 0xD4
 	ContactRegroup = 0xE7
+	NewBuddyList = 0xF1
 	# Documented by the Yahsmosis project (https://www.autoitscript.com/forum/topic/142448-help-with-yahomosis/) - this appears to be used when a protocol-level error occurs (protocol version "cloaking", bad `Y`/`T` cookies, invalid Yahoo ID, invalid key/value pairs, etc.). Unsure what protocol version this was first placed into or how far back in the protocol version Yahoo's servers decided to send this on (A Wireshark capture indicates as far back as YMSG12)
 	ProtocolError = 0x07D1
 

front/ymsg/pager.py

@@ -1,11 +1,11 @@
 from typing import Optional, Dict, Any, Tuple
 import datetime, asyncio, time, settings, base64, util, secrets, json
-
+ 
 from urllib.parse import parse_qs
 from disposable_email_domains import blocklist as disposable_emails
-
+ 
 from util.misc import Logger, gen_uuid, MultiDict, arbitrary_decode, arbitrary_encode
-
+ 
 from core import event, error
 from core.backend import Backend, BackendSession, Chat, ChatSession
 from core.models import (
@@ -14,15 +14,15 @@ from core.models import (
 )
 from core.client import Client
 from core.auth import GenTokenStr
-
+ 
 from .ymsg_ctrl import YMSGCtrlBase
 from .misc import YMSGService, YMSGStatus, split_to_chunks
 from . import misc, Y64
-
+ 
 # "Pre" because it's needed before BackendSession is created.
 # Comment out session ID stuff as that'll have to be reworked with account usernames being a thing
 #PRE_SESSION_ID: Dict[str, int] = {}
-
+ 
 class YMSGCtrlPager(YMSGCtrlBase):
 	__slots__ = ('backend', 'dialect', 'sess_id', 'challenge', 't_cookie_token', 'bs', 'client')
 	
@@ -60,11 +60,12 @@ class YMSGCtrlPager(YMSGCtrlBase):
 	def _y_0057(self, *args: Any) -> None:
 		# SERVICE_AUTH (0x57); send a challenge string for the client to craft two response strings with
 		backend = self.backend
-
+ 
 		# Yahoo for Vista & 9+ doesn't send SERVICE_HANDSHAKE apparently, so I'm adding this here as well
 		self.client = Client('ymsg', 'YMSG{}'.format(str(args[0])), self.client.via)
 		self.dialect = int(args[0])
 		
+		print(f'Shit: {self.dialect}')
 		yahoo_data = args[4] # type: MultiDict[bytes, bytes]
 		
 		yahoo_id = yahoo_data.get(b'1')
@@ -76,7 +77,7 @@ class YMSGCtrlPager(YMSGCtrlBase):
 		]) # type: MultiDict[bytes, bytes]
 		if settings.STRESS_TEST_ACTIVE:
 			send_system_message(self, "We're currently doing a stress test of the CrossTalk service. Put load on the server, and tell your friends!")
-
+ 
 		if settings.FORCE_HOSTS_UPDATE:
 			send_system_message(self, "You are required to update your patches before continuing. Go here to learn more: http://diet.crosstalk.im/moving-hosts")
 			
@@ -89,11 +90,11 @@ class YMSGCtrlPager(YMSGCtrlBase):
 		else:
 			auth_dict.add(b'13', b'2')
 			self.challenge = base64.b64encode(util.misc.generate_random_string(chars=48))
-
+ 
 		auth_dict.add(b'94', self.challenge)			
-
+ 
 		self.send_reply(YMSGService.Auth, YMSGStatus.BRB, self.sess_id, auth_dict)
-
+ 
 	def _y_0001(self, *args: Any) -> None:
 		backend = self.backend
 		
@@ -133,7 +134,7 @@ class YMSGCtrlPager(YMSGCtrlBase):
 					t = arbitrary_decode(yahoo_data.get(b'278'))
 					self.logger.debug(f'Y Cookie sent by client: {y}')
 					self.logger.debug(f'T Cookie sent by client: {t}')
-
+ 
 					token = None
 					if t:
 						qs = parse_qs(t, keep_blank_values=True, strict_parsing=False)
@@ -143,9 +144,9 @@ class YMSGCtrlPager(YMSGCtrlBase):
 							token = qs['sk'][0]
 						elif 'd' in qs and qs['d']:
 							token = qs['d'][0]
-
+ 
 					if not token:
-						self.logger.error('Could not extract token from T cookie')
+						self.logger.error('Could not extract tokenfrom T cookie')
 						authresp_error = int(YMSGStatus.Bad)
 					else:
 						tpl = self.backend.login_auth_service.get_token('ymsg/cookie', token)
@@ -180,7 +181,7 @@ class YMSGCtrlPager(YMSGCtrlBase):
 				(b'0', yahoo_id),
 			]))
 			self.close()
-
+ 
 	def _y_0054(self, *args: Any) -> None:
 		backend = self.backend
 		
@@ -220,7 +221,7 @@ class YMSGCtrlPager(YMSGCtrlBase):
 					t = arbitrary_decode(yahoo_data.get(b'278'))
 					self.logger.debug(f'Y Cookie sent by client: {y}')
 					self.logger.debug(f'T Cookie sent by client: {t}')
-
+ 
 					token = None
 					if t:
 						qs = parse_qs(t, keep_blank_values=True, strict_parsing=False)
@@ -230,7 +231,7 @@ class YMSGCtrlPager(YMSGCtrlBase):
 							token = qs['sk'][0]
 						elif 'd' in qs and qs['d']:
 							token = qs['d'][0]
-
+ 
 					if not token:
 						self.logger.info('Could not extract token from T cookie')
 						authresp_error = int(YMSGStatus.Bad)
@@ -267,7 +268,7 @@ class YMSGCtrlPager(YMSGCtrlBase):
 				(b'0', yahoo_id),
 			]))
 			self.close()
-
+ 
 	def _util_authresp_final(self, status: YMSGStatus, *, cached_y: Optional[str] = None, cached_t: Optional[str] = None) -> None:
 		bs = self.bs
 		assert bs is not None
@@ -279,15 +280,21 @@ class YMSGCtrlPager(YMSGCtrlBase):
 		bs.front_data['ymsg'] = True
 		bs.front_data['ymsg_private_chats'] = {}
 		bs.front_data['ymsg_chat_sessions'] = {}
-
+ 
 		if user.suspended:
 			send_system_message(self, "Your CrossTalk account has been suspended. You may not connect to the service.")
 			
 			authresp_error = int(YMSGStatus.Bad)
 			self.close()
 			return
-
-
+ 
+		#if not user.verified_to_login:
+		#	send_system_message(self, "You need to verify the e-mail address attached to this account belongs to you before you can log in. Log in to the CrossTalk site and do so in your Account Settings.")
+		#	
+		#	authresp_error = int(YMSGStatus.Bad)
+		#	self.close()
+		#	return
+ 
 		with open('config/restricted-emails.json', 'r') as file:
 			restricted_emails = json.load(file)
 			email_domain = email.lower().split('@')[-1]
@@ -311,7 +318,7 @@ class YMSGCtrlPager(YMSGCtrlBase):
 		
 		if self.backend.notify_maintenance:
 			bs.evt.on_maintenance_message(None, self.backend.maintenance_mins)
-
+ 
 	# State = Live
 	
 	def _y_0004(self, *args: Any) -> None:
@@ -538,7 +545,7 @@ class YMSGCtrlPager(YMSGCtrlBase):
 		self._update_buddy_list()
 	
 	def _y_0084(self, *args: Any) -> None:
-		# SERVICE_FRIENDREMOVE (0x84); remove a buddy from your list
+		# SERVICE_FRIENDREMOVE (0x84); remove a buddyfrom your list
 		backend = self.backend
 		
 		yahoo_data = args[4] # type: MultiDict[bytes, bytes]
@@ -591,7 +598,7 @@ class YMSGCtrlPager(YMSGCtrlBase):
 		self._update_buddy_list()
 	
 	def _y_0085(self, *args: Any) -> None:
-		# SERVICE_IGNORE (0x85); add/remove someone from your ignore list
+		# SERVICE_IGNORE (0x85); add/remove someonefrom your ignore list
 		
 		yahoo_data = args[4] # type: MultiDict[bytes, bytes]
 		
@@ -648,27 +655,30 @@ class YMSGCtrlPager(YMSGCtrlBase):
 			YMSGService.UserStat, bs.front_data.get('ymsg_status') or YMSGStatus.FromSubstatus(user.status.substatus),
 			self.sess_id, self._gen_multi_user_status_dict(user),
 		)
-
+ 
 	def _y_00d2(self, *args: Any) -> None:
 		# photo sharing invite, stub for now
 		pass
-
+ 
 	def _y_00dc(self, *args: Any) -> None:
 		# P2P file transfers on newer version, stub for now
 		pass
 	
 	def _y_00bd(self, *args: Any) -> None:
 		pass
-
+ 
+	def _y_00c1(self, *args: Any) -> None:
+		pass
+	
 	def _y_00c7(self, *args: Any) -> None:
 		# for changing avatars, ignore for now
 		pass
-
+ 
 	def _y_0055(self, *args: Any) -> None:
 		# SERVICE_LIST (0x55); send a user's buddy list
 		
 		self._update_buddy_list(after_login = True)
-
+ 
 	def _y_00c5(self, *args: any) -> None:
 		# SERVICE_ISINVISIBLE (0xc5); mark as invisible
 		bs = self.bs
@@ -682,12 +692,12 @@ class YMSGCtrlPager(YMSGCtrlBase):
 		elif new_status == int(1):
 			status = YMSGStatus.Available
 		me_status_update(bs, status)
-  
+   
 	def _y_00c6(self, *args: Any) -> None:
 		# SERVICE_STATUSUPDATE (0xc6); essentially the same as 0x03. For whatever reason, it was changed in YMSG12
 		bs = self.bs
 		assert bs is not None
-  
+   
 		yahoo_data = args[4]
 		new_status = YMSGStatus(int(yahoo_data.get(b'10') or b''))
 		message = arbitrary_decode(yahoo_data.get(b'19') or b'')
@@ -757,64 +767,64 @@ class YMSGCtrlPager(YMSGCtrlBase):
 				cs.send_message_to_everyone(messagedata_from_ymsg(cs.user, yahoo_data, notify_type = notify_type, typing_flag = typing_flag))
 		except error.ContactNotOnline:
 			pass
-
+ 
 	def _y_00d4(self, *args: Any) -> None:
 	# SERVICE_CHATSESSION (0xD4); start a chat session
 		pass
-
+ 
 	def _y_00e7(self, *args: Any) -> None:
 		# gotta figure out what the fuck this one is, but it's used in YMSG14/15 for group management
 		backend = self.backend
 		bs = self.bs
 		assert bs is not None
 		user = bs.user
-
+ 
 		yahoo_data = args[4] # type: MultiDict[bytes, bytes]
-
+ 
 		yahoo_id = arbitrary_decode(yahoo_data.get(b'1') or b'')
 		contact_yahoo_id = arbitrary_decode(yahoo_data.get(b'7') or b'')
 		new_buddy_group = arbitrary_decode(yahoo_data.get(b'264') or b'')
 		old_buddy_group = arbitrary_decode(yahoo_data.get(b'224') or b'')
-
+ 
 		group_new = None
 		group_old = None
 		action_group_refresh = False
-
+ 
 		contact_uuid = backend.util_get_uuid_from_username(contact_yahoo_id)
 		if contact_uuid is None:
 			return
-
+ 
 		bs = self.bs
 		assert bs is not None
 		user = bs.user
 		detail = user.detail
 		assert detail is not None
-
+ 
 		contacts = detail.contacts
-
+ 
 		contact = contacts.get(contact_uuid)
 		if contact is None:
 			return
-
+ 
 		if new_buddy_group != '(No Group)':
 			for grp in detail._groups_by_id.values():
 				if grp.name == new_buddy_group:
 					group_new = grp
 					break
-
+ 
 			if group_new is None:
 				group_new = bs.me_group_add(new_buddy_group)
-
+ 
 		if old_buddy_group and old_buddy_group != '(No Group)':
 			for grp in detail._groups_by_id.values():
 				if grp.name == old_buddy_group:
 					group_old = grp
 					break
-
+ 
 		try:
 			if len(contact._groups) >= 1 or (contact._groups and new_buddy_group == '(No Group)'):
 				action_group_refresh = True
-
+ 
 			if old_buddy_group == '(No Group)':
 				for group_other in contact._groups.copy():
 					group_full = False
@@ -830,15 +840,15 @@ class YMSGCtrlPager(YMSGCtrlBase):
 			else:
 				if group_old is not None:
 					bs.me_group_contact_remove(group_old.id, contact.head.uuid)
-
+ 
 			if group_new is not None:
 				bs.me_group_contact_add(group_new.id, contact.head.uuid)
-
+ 
 			if action_group_refresh:
 				self._update_buddy_list()
 		except error.ContactAlreadyOnContactList:
 			pass
-
+ 
 	
 	def _y_0006(self, *args: Any) -> None:
 		# SERVICE_MESSAGE (0x06); send a message to a user (older YMSG)
@@ -1071,7 +1081,7 @@ class YMSGCtrlPager(YMSGCtrlBase):
 				yhlper_msg_dict.add(b'64', yahoo_data.get(b'64') or b'')
 			
 			yhlper_msg_dict.add(b'97', b'1')
-
+ 
 			if yahoo_data.get(b'206') is not None:
 				yhlper_msg_dict.add(b'206', yahoo_data.get(b'206') or b'')
 			
@@ -1110,7 +1120,7 @@ class YMSGCtrlPager(YMSGCtrlBase):
 				
 				self.backend.user_service.save_oim(
 					bs, contact_uuid, gen_uuid(), ip, md.text or '', False if md.front_cache['ymsg'].get(b'97') == b'0' else True,
-					from_user_id = from_user_id,
+					from_user_id =from_user_id,
 				)
 		except error.ContactNotOnContactList:
 			pass
@@ -1175,10 +1185,78 @@ class YMSGCtrlPager(YMSGCtrlBase):
 		assert detail is not None
 		
 		contacts = detail.contacts
-		
 		cs = list(contacts.values())
 		cs_fl = [c for c in cs if c.lists & ContactList.FL and not c.lists & ContactList.BL]
+		ignore_list = [c for c in cs if c.lists & ContactList.BL]
 		
+		if self.dialect >= 16:
+			self.generate_buddy_list_assnew(user, detail, cs_fl, ignore_list)
+		else:
+			self.generate_buddy_list_assold(backend, user, detail, cs_fl, ignore_list, cached_y, cached_t)
+		
+		if on_login:
+			me_status_update(bs, YMSGStatus.Available)
+		
+		if not after_login:
+			self.send_reply(YMSGService.LogOn, YMSGStatus.Available, self.sess_id, self._gen_multi_user_status_dict(user))
+	
+	def generate_buddy_list_assnew(self, user: User, detail: Any, cs_fl: list, ignore_list: list) -> None:
+		kvs = MultiDict() # type: MultiDict[bytes, bytes]
+		kvs.add(b'1', arbitrary_encode(user.username))
+		kvs.add(b'4', arbitrary_encode(user.username))
+
+		kvs.add(b'302', b'318')
+
+		for grp in detail._groups_by_id.values():
+			buddies_in_group = []
+			for c in cs_fl:
+				for group in c._groups.copy():
+					if group.id == grp.id:
+						buddies_in_group.append(c)
+			kvs.add(b'300', b'318')
+			kvs.add(b'65', arbitrary_encode(grp.name))
+			kvs.add(b'302', b'319')
+			for c in buddies_in_group:
+				kvs.add(b'300', b'319')
+				kvs.add(b'241', b'0')
+				kvs.add(b'7', arbitrary_encode(c.head.username))
+				kvs.add(b'317', b'1')
+				kvs.add(b'301', b'319')
+			kvs.add(b'303', b'319')
+			kvs.add(b'301', b'318')
+
+		no_group_buddies = [c for c in cs_fl if not c._groups]
+		if no_group_buddies:
+			kvs.add(b'300', b'318')
+			kvs.add(b'65', b'(No Group)')
+			kvs.add(b'302', b'319')
+			for c in no_group_buddies:
+				kvs.add(b'300', b'319')
+				kvs.add(b'241', b'0')
+				kvs.add(b'7', arbitrary_encode(c.head.username))
+				kvs.add(b'317', b'1')
+				kvs.add(b'301', b'319')
+			kvs.add(b'303', b'319')
+			kvs.add(b'301', b'318')
+
+		kvs.add(b'303', b'318')
+
+		kvs.add(b'302', b'320')
+		for c in ignore_list:
+			kvs.add(b'300', b'320')
+			kvs.add(b'7', arbitrary_encode(c.head.username))
+			kvs.add(b'301', b'320')
+		kvs.add(b'303', b'320')
+
+		# Service 0xF1 (241) is the YMSG16 structured buddy list command.
+		self.send_reply(YMSGService(0xF1), YMSGStatus.Available, self.sess_id, kvs)
+	
+	def generate_buddy_list_assold(
+		self, backend: Backend, user: User, detail: Any,
+		cs_fl: list, ignore_list: list,
+		cached_y: Optional[str], cached_t: Optional[str],
+	) -> None:
+		# YMSG <= 15: flat text encoding on keys 87 (contacts) and 88 (ignored).
 		contact_group_list = []
 		for grp in detail._groups_by_id.values():
 			contact_list = []
@@ -1189,20 +1267,17 @@ class YMSGCtrlPager(YMSGCtrlBase):
 			if contact_list:
 				contact_group_list.append(grp.name + ':' + ','.join(contact_list) + '\n')
 		# Handle contacts that aren't part of any groups
-		contact_list = [c.head.username for c in cs_fl if not c._groups]
-		if contact_list:
-			contact_group_list.append('(No Group):' + ','.join(contact_list) + '\n')
+		no_group = [c.head.username for c in cs_fl if not c._groups]
+		if no_group:
+			contact_group_list.append('(No Group):' + ','.join(no_group) + '\n')
 		
 		contact_list_format = ''.join(contact_group_list)
-		
-		ignore_list = [c.head.username for c in cs if c.lists & ContactList.BL]
-		ignore_list_format = ','.join(ignore_list)
+		ignore_list_format = ','.join(c.head.username for c in ignore_list)
 		
 		list_reply_kvs = MultiDict() # type: MultiDict[bytes, bytes]
 		
 		if len(contact_list_format) > 815:
-			contact_chunks = split_to_chunks(contact_list_format, 815)
-			for contact_chunk in contact_chunks:
+			for contact_chunk in split_to_chunks(contact_list_format, 815):
 				self.send_reply(YMSGService.List, YMSGStatus.NotInOffice, self.sess_id, MultiDict([
 					(b'87', arbitrary_encode(contact_chunk)),
 				]))
@@ -1210,8 +1285,7 @@ class YMSGCtrlPager(YMSGCtrlBase):
 			list_reply_kvs.add(b'87', arbitrary_encode(contact_list_format))
 		
 		if len(ignore_list_format) > 815:
-			ignore_chunks = split_to_chunks(ignore_list_format, 815)
-			for ignore_chunk in ignore_chunks:
+			for ignore_chunk in split_to_chunks(ignore_list_format, 815):
 				self.send_reply(YMSGService.List, YMSGStatus.NotInOffice, self.sess_id, MultiDict([
 					(b'88', ignore_chunk.encode('utf-8')),
 				]))
@@ -1234,7 +1308,7 @@ class YMSGCtrlPager(YMSGCtrlBase):
 				list_reply_kvs.add(b'59', b'T\t')
 			else:
 				(y_cookie, t_cookie, y_expiry, t_expiry) = tpl
-				domain = (settings.TARGET_HOST)
+				domain = settings.TARGET_HOST
 				list_reply_kvs.add(b'59', arbitrary_encode('Y\t{}; expires={}; path=/; domain={}'.format(y_cookie, y_expiry, domain)))
 				list_reply_kvs.add(b'59', arbitrary_encode('T\t{}; expires={}; path=/; domain={}'.format(t_cookie, t_expiry, domain)))
 		
@@ -1247,12 +1321,6 @@ class YMSGCtrlPager(YMSGCtrlBase):
 		list_reply_kvs.add(b'93', b'86400')
 		
 		self.send_reply(YMSGService.List, YMSGStatus.Available, self.sess_id, list_reply_kvs)
-		
-		if on_login:
-			me_status_update(bs, YMSGStatus.Available)
-		
-		if not after_login:
-			self.send_reply(YMSGService.LogOn, YMSGStatus.Available, self.sess_id, self._gen_multi_user_status_dict(user))
 	
 	def _gen_multi_user_status_dict(self, user: User) -> MultiDict[bytes, bytes]:
 		detail = user.detail
@@ -1276,7 +1344,7 @@ class YMSGCtrlPager(YMSGCtrlBase):
 		two_users = False
 	
 		while True:
-			await asyncio.sleep(0.0125)
+			await asyncio.sleep(0.04)
 		
 			if 'ymsg_twoway_only' in chat.front_data:
 				if len(list(chat.get_roster_single())) == 2:
@@ -1323,7 +1391,7 @@ class YMSGCtrlPager(YMSGCtrlBase):
 			oim_msg_dict.add(b'14', arbitrary_encode(oim.message))
 			oim_msg_dict.add(b'15', str(int(oim.sent.timestamp())).encode('utf-8'))
 			oim_msg_dict.add(b'97', b'1' if oim.utf8 else b'0')
-	   
+    
 			self.backend.user_service.delete_oim(user.uuid, oim.uuid)
 	
 			if len(list(oim_msg_dict.items())) > 0:
@@ -1351,10 +1419,10 @@ class YMSGCtrlPager(YMSGCtrlBase):
 		if uuid is None:
 			return False
 		
-		# Retrieve Yahoo64-encoded MD5 hash of the user's password from the database
-		# NOTE: The MD5 hash of the password is unsalted.
+		# Retrieve Yahoo64-encoded MD5 hash of the user's passwordfrom the database
+		# NOTE: The MD5 hash of the password is unsa<ed.
 		pass_md5 = Y64.Y64Encode(self.backend.user_service.yahoo_get_md5_password(uuid) or b'')
-		# Retrieve MD5-crypt(3)'d hash of the user's password from the database
+		# Retrieve MD5-crypt(3)'d hash of the user's passwordfrom the database
 		pass_md5crypt = Y64.Y64Encode(md5(self.backend.user_service.yahoo_get_md5crypt_password(uuid) or b'').digest())
 		
 		seed_val = (chal[15] % 8) % 5
@@ -1397,10 +1465,10 @@ class YMSGCtrlPager(YMSGCtrlBase):
 		
 		y_cookie = Y_COOKIE_TEMPLATE.format(encodedname=_encode_yahoo_id(user.username))
 		t_cookie = T_COOKIE_TEMPLATE.format(token=self.t_cookie_token)
-
+ 
 		y_expiry = 0
 		t_expiry = 0
-
+ 
 		try:
 			_, y_expiry = auth_service.create_token('ymsg/cookie', user.username, token=y_cookie, lifetime=86400)
 			_, t_expiry = auth_service.create_token('ymsg/cookie', self.bs, token=t_cookie, lifetime=86400)
@@ -1413,21 +1481,20 @@ class YMSGCtrlPager(YMSGCtrlBase):
 			_format_cookie_expiry(datetime.datetime.utcfromtimestamp(y_expiry)),
 			_format_cookie_expiry(datetime.datetime.utcfromtimestamp(t_expiry))
 		)
-
+ 
 Y_COOKIE_TEMPLATE = 'v=1&n=&l={encodedname}&p=&r=&lg=&intl=&np='
-T_COOKIE_TEMPLATE = 'z={token}&a=&sk={token}&ks={token}&kt=&ku=&d={token}'
-
+T_COOKIE_TEMPLATE = 'z={token}&a=&sk={token}&ks={token}&kt=&ku=&d={token}' 
 YAHOO_HELPER_MSG = settings.YAHOOHELPER_MSG
-
+ 
 def _format_cookie_expiry(expiry: datetime.datetime) -> str:
 	return expiry.strftime('%a, %d %b %Y %H:%M:%S GMT')
-
+ 
 def _encode_yahoo_id(yahoo_id: str) -> str:
 	return ''.join(
 		YAHOO_ID_ENCODING.get(c) or c
 		for c in yahoo_id
 	)
-
+ 
 YAHOO_ID_ENCODING = {
 	'k': 'a',
 	'l': 'b',
@@ -1466,12 +1533,12 @@ YAHOO_ID_ENCODING = {
 	'i': '8',
 	'j': '9',
 }
-
+ 
 def add_contact_status_to_data(
 	data: Any, status: UserStatus, contact: User, *,
 	old_substatus: Substatus = Substatus.Offline, message: Optional[str] = None,
 	exclude_psm: bool = False, sess_id: Optional[int] = None,
-) -> None:
+ ) -> None:
 	is_offlineish = status.is_offlineish()
 	# `static var YMSG_FLD_SESSION_ID = 11;`
 	# Yahoo! was weird sometimes :p
@@ -1501,7 +1568,7 @@ def add_contact_status_to_data(
 	
 	data.add(b'17', b'0')
 	data.add(b'13', (b'0' if is_offlineish else b'1'))
-
+ 
 class BackendEventHandler(event.BackendEventHandler):
 	__slots__ = ('loop', 'ctrl', 'dialect', 'sess_id', 'bs')
 	
@@ -1520,13 +1587,13 @@ class BackendEventHandler(event.BackendEventHandler):
 	def on_maintenance_message(self, *args: Any, **kwargs: Any) -> None:
 		if args[1] is not None and args[1] > 0:
 			msg = """CrossTalk will be down for maintenance in around {} minute(s). \
-Now is a good time to wrap up any conversations.""".format(args[1])
+ Now is a good time to wrap up any conversations.""".format(args[1])
 		kvs = MultiDict([
 			(b'14', msg.encode('utf-8')),
 			(b'15', str(time.time()).encode('utf-8')),
 		]) # type: MultiDict[bytes, bytes]
 		self.ctrl.send_reply(YMSGService.SystemMessage, YMSGStatus.BRB, self.sess_id, kvs)
-
+ 
 	def on_client_alert(self, icon_url: Optional[str], url: Optional[str], message: str = '') -> None:
 		msg = message
 		kvs = MultiDict([
@@ -1613,7 +1680,7 @@ Now is a good time to wrap up any conversations.""".format(args[1])
 			(b'63', b';0'),
 			(b'64', b'0'),
 			(b'97', (b'1' if oim.utf8 else b'0')),
-		]) 	# type: MultiDict[bytes, bytes]
+		])	# type: MultiDict[bytes, bytes]
 	
 		self.ctrl.send_reply(YMSGService.Message, YMSGStatus.BRB, self.ctrl.sess_id, message_dict)
 	
@@ -1665,7 +1732,7 @@ Now is a good time to wrap up any conversations.""".format(args[1])
 		
 		if circle:
 			return
-
+ 
 		roster = list(chat.get_roster_single())
 		
 		if 'ymsg_twoway_only' not in chat.front_data and len(roster) == 2:
@@ -1674,7 +1741,7 @@ Now is a good time to wrap up any conversations.""".format(args[1])
 			del chat.front_data['ymsg_twoway_only']
 			if 'ymsg/conf' not in chat.ids:
 				chat.add_id('ymsg/conf', chat.ids['main'])
-
+ 
 		if chat.front_data.get('ymsg_twoway_only') or len(roster) < 2:
 			# A Yahoo! non-conference chat; auto-accepted invite
 			evt = ChatEventHandler(self.loop, self.ctrl, self.bs)
@@ -1744,7 +1811,7 @@ Now is a good time to wrap up any conversations.""".format(args[1])
 	
 	def on_close(self) -> None:
 		self.ctrl.close()
-
+ 
 class ChatEventHandler(event.ChatEventHandler):
 	__slots__ = ('loop', 'ctrl', 'bs', 'cs')
 	
@@ -1900,10 +1967,10 @@ class ChatEventHandler(event.ChatEventHandler):
 			if cs_other.user.uuid == user_uuid:
 				return True
 		return False
-
+ 
 def messagedata_from_ymsg(
 	sender: User, data: MultiDict[bytes, bytes], *, notify_type: Optional[bytes] = None, typing_flag: Optional[str] = None,
-) -> MessageData:
+ ) -> MessageData:
 	text = arbitrary_decode(data.get(b'14') or b'')
 	
 	if notify_type is None:
@@ -1926,7 +1993,7 @@ def messagedata_from_ymsg(
 	message = MessageData(sender = sender, type = type, text = text)
 	message.front_cache['ymsg'] = data
 	return message
-
+ 
 def messagedata_to_ymsg(data: MessageData) -> MultiDict[bytes, bytes]:
 	if 'ymsg' not in data.front_cache:
 		data.front_cache['ymsg'] = MultiDict([
@@ -1937,7 +2004,7 @@ def messagedata_to_ymsg(data: MessageData) -> MultiDict[bytes, bytes]:
 			(b'206', b'2')
 		])
 	return data.front_cache['ymsg']
-
+ 
 def me_status_update(bs: BackendSession, status_new: YMSGStatus, *, message: str = '', is_away_message: bool = False) -> None:
 	bs.front_data['ymsg_status'] = status_new
 	if status_new is YMSGStatus.Custom:
@@ -1950,7 +2017,7 @@ def me_status_update(bs: BackendSession, status_new: YMSGStatus, *, message: str
 		'notify_status': True,
 		'notify_info': (True if message else False),
 	})
-
+ 
 def send_system_message(self, message):
 	msg = f"{message}"
 	kvs = MultiDict([
@@ -1958,9 +2025,9 @@ def send_system_message(self, message):
 		(b'15', str(time.time()).encode('utf-8')),
 	]) # type: MultiDict[bytes, bytes]
 	self.send_reply(YMSGService.SystemMessage, YMSGStatus.BRB, self.sess_id, kvs)
-
+ 
 def generate_challenge_v1() -> bytes:
 	from uuid import uuid4
 	
 	# Yahoo64-encode the raw 16 bytes of a UUID
-	return Y64.Y64Encode(uuid4().bytes)
+	return Y64.Y64Encode(uuid4().bytes)

front/ymsg/tmpl/layout/nofriends.html

@@ -0,0 +1,7 @@
+<html>
+<head>
+</head>
+<body>
+<h1>i'm gay</h1>
+</body>
+</html>

run_all.py

@@ -7,12 +7,32 @@ from core.user import UserService
 from core.stats import Stats
 
 def main(*, devmode: bool = False) -> None:
+	
 	sys.excepthook = _excepthook
 
-	import asyncio, settings
+	import asyncio, settings, platform
 	from core.backend import Backend
 	from core import http
 
+	print("""   █████████                         ████ 
+  ███▒▒▒▒▒███                       ▒▒███ 
+ ▒███    ▒███   █████████ █████ ████ ▒███ 
+ ▒███████████  ▒█▒▒▒▒███ ▒▒███ ▒███  ▒███ 
+ ▒███▒▒▒▒▒███  ▒   ███▒   ▒███ ▒███  ▒███ 
+ ▒███    ▒███    ███▒   █ ▒███ ▒███  ▒███ 
+ █████   █████  █████████ ▒▒████████ █████
+▒▒▒▒▒   ▒▒▒▒▒  ▒▒▒▒▒▒▒▒▒   ▒▒▒▒▒▒▒▒ ▒▒▒▒▒
+
+""")
+
+	print(f"===== AZUL {settings.VERSION} =====")
+	print(f"""2023 - 2026 the undergr0und
+	""")
+	print(f"""Running on Python {sys.version} on {platform.platform()}
+	""")
+	if settings.DEBUG:
+		print(f"""Debug mode on!
+		""")
 	loop = asyncio.new_event_loop()
 	asyncio.set_event_loop(loop)
 

settings.py

@@ -18,6 +18,10 @@ ADDRESSBOOK_HOST = 'ctsvcs.addressbook.ugnet.gay'
 USERSTORAGE_HOST = 'cts.storage.ugnet.gay'
 STORAGE_HOST = 'storage.ugnet.gay'
 STATIC_HOST = 'static.ugnet.gay'
+AD_HOST = 'ctsvcs.advertising.ugnet.gay'
+TODAY_HOST = 'today.msgrsvcs.ctsrv.gay'
+SOCIAL_HOST = 'social.ugnet.gay'
+WEBSITE_HOST = 'crosstalk.im'
 TARGET_IP = '127.0.0.1'
 SMTP_HOST = 'mail-server-here'
 SMTP_PORT = 587
@@ -25,7 +29,7 @@ SMTP_USERNAME = 'administration@ugnet.gay'
 SMTP_PASSWORD = ''
 ENABLE_NAT_RELAY = False
 CF_REALTIME_API_KEY = ''
-VERSION = '0.5.23'
+VERSION = '0.6.0-dp2'
 # While not necessary for debugging, it is recommended you change the password variables in production for obvious security reasons.
 INS_LINK_PASSWORD = 'password'
 STRESS_TEST_ACTIVE = False

util/avatar.py

@@ -0,0 +1,107 @@
+from __future__ import annotations
+
+import hashlib
+from io import BytesIO
+from pathlib import Path
+from typing import Optional, Tuple
+
+STORAGE_BASE = Path('storage/dp')
+THUMB_SIZE = (21, 21)
+MSN_SIZE   = (96, 96)
+1
+def compute_md5(data: bytes) -> str:
+	return hashlib.md5(data).hexdigest()[:16]
+
+def get_img_type(data: bytes) -> Optional[str]:
+	if data[:6] in (b'GIF87a', b'GIF89a'):
+		return 'gif'
+	if data[:2] == b'\xff\xd8':
+		return 'jpeg'
+	if data[:8] == b'\x89PNG\r\n\x1a\n':
+		return 'png'
+	return None
+
+def get_avatar_dir(uuid: str) -> Path:
+	return STORAGE_BASE / uuid
+
+def store_avatar_file(uuid: str, data: bytes, ext: str) -> str:
+	from PIL import Image
+
+	md5 = compute_md5(data)
+	avatar_dir = get_avatar_dir(uuid)
+	avatar_dir.mkdir(parents=True, exist_ok=True)
+
+	full_path  = avatar_dir / f'{md5}.{ext}'
+	thumb_path = avatar_dir / f'{md5}_thumb.png'
+	msn_path   = avatar_dir / f'{md5}_msn.png'
+
+	if not full_path.exists():
+		full_path.write_bytes(data)
+
+	if not thumb_path.exists() or not msn_path.exists():
+		try:
+			img = Image.open(BytesIO(data))
+			if not thumb_path.exists():
+				img.resize(THUMB_SIZE).save(str(thumb_path))
+			if not msn_path.exists():
+				img.resize(MSN_SIZE).save(str(msn_path))
+		except Exception:
+			pass 
+
+	return md5
+
+
+def find_avatar_file(
+	uuid: str,
+	md5: str,
+	*,
+	small: bool = False,
+	msn: bool = False,
+) -> Optional[Path]:
+	avatar_dir = get_avatar_dir(uuid)
+	if not avatar_dir.is_dir():
+		return None
+
+	if small:
+		thumb = avatar_dir / f'{md5}_thumb.png'
+		if thumb.is_file():
+			return thumb
+
+	elif msn:
+		msn_file = avatar_dir / f'{md5}_msn.png'
+		if msn_file.is_file():
+			return msn_file
+
+	for p in avatar_dir.iterdir():
+		if p.is_file() and p.stem == md5:
+			return p
+
+	return None
+
+
+def get_avatar_data(
+	uuid: str,
+	md5: Optional[str],
+	*,
+	small: bool = False,
+	msn: bool = False,
+) -> Optional[Tuple[bytes, str]]:
+	if not md5:
+		return None
+
+	path = find_avatar_file(uuid, md5, small=small, msn=msn)
+	if path is None:
+		return None
+
+	try:
+		data = path.read_bytes()
+	except OSError:
+		return None
+
+	if not data:
+		return None
+
+	ext = path.suffix.lstrip('.').lower() or 'png'
+	mime_map = {'jpg': 'jpeg', 'jpeg': 'jpeg', 'png': 'png', 'gif': 'gif'}
+	content_type = 'image/' + mime_map.get(ext, ext)
+	return data, content_type

util/hash.py

@@ -120,6 +120,17 @@ class MD5PasswordHasher(Hasher):
 
 		return md5.digest(), []
 
+	# maybe we could find a better way to do this? it works for now though
+	@classmethod
+	def encode_aim5(cls, password: str, *, salt: str) -> str:
+		md5_pw_raw = hashlib.md5(password.encode('utf-8')).digest()
+		md5 = hashlib.md5()
+		md5.update(salt.encode('utf-8'))
+		md5.update(md5_pw_raw)
+		md5.update(b'AOL Instant Messenger (SM)')
+		hash_b64 = base64.b64encode(md5.digest()).decode('ascii').strip()
+		return cls.separator.join([cls.algorithm, salt, hash_b64])
+
 	@classmethod
 	def verify_hash(cls, hash_1: str, encoded: str) -> bool:
 		try:

util/misc.py

@@ -132,25 +132,7 @@ class Logger:
 		self.debug("Disconnected!")
 
 def run_loop(loop: asyncio.AbstractEventLoop, runners: List[Runner]) -> None:
-	print("""   █████████                         ████ 
-  ███▒▒▒▒▒███                       ▒▒███ 
- ▒███    ▒███   █████████ █████ ████ ▒███ 
- ▒███████████  ▒█▒▒▒▒███ ▒▒███ ▒███  ▒███ 
- ▒███▒▒▒▒▒███  ▒   ███▒   ▒███ ▒███  ▒███ 
- ▒███    ▒███    ███▒   █ ▒███ ▒███  ▒███ 
- █████   █████  █████████ ▒▒████████ █████
-▒▒▒▒▒   ▒▒▒▒▒  ▒▒▒▒▒▒▒▒▒   ▒▒▒▒▒▒▒▒ ▒▒▒▒▒
 
-""")
-
-	print(f"===== Azul {settings.VERSION} =====")
-	print(f"""2023 - 2026 the undergr0und
-	""")
-	print(f"""Running on Python {sys.version} on {platform.platform()}
-	""")
-	if settings.DEBUG:
-		print(f"""Debug mode on!
-		""")
 	for runner in runners:
 		print("[{}] Started service on {}:{}".format(runner.service, runner.host, runner.port))