A list of discoveries in the development of Giraffael

Database

Database Considerations for Giraffael

• Relational Databases (SQL): Diffrences
  • MySQL: good performance for web applications, but can be less feature-rich than PostgreSQL, and potential performance issues with complex queries.
  • PostgreSQL: excellent for complex data relationships and high performance, but can be overkill for simple projects.
  • SQLite: great for small projects or prototyping, but not ideal for large datasets or high concurrency.
• No SQL Databases:
  • MongoDB: great for unstructured data, but can be less efficient for complex queries, with less emphasis on data integrity.
• MongoDB vs. Redis:
  • MongoDB for permanently stored stuffs like user profiles, rooms, messages and files.
  • Redis (Remote Dictionary Server) for frequently real-time updated/retrieved/called stuffs like presence/heartbeat, cross-server broadcast, typing indicator, unread msg count, and other frequently-used data that should be cached.
    • Redis is an external memory database that is mainly used to fast cache results (useful for cases where some APIs or distributed services can be “stateless” to be scalable with load balancing).
    • Redis also provides inter-process data sharing, where it allows multiple workers across multiple machines to share some needed data. It stores the data in RAM instead of on disk, which can perform faster and less overkill (it can have latency brought by it transmitting through the Network layer).

Other Findings

• Since there is no an “empty message” in TCP, server can test whether a client disconnected with something like this:

	data = conn.recv(1024)
  if not data: break

• WebSocket vs. FastAPI:
  • WebSocket: used for real-time message communication
    • Message transmission, type indicator, msg 'read' indicator, etc.
    • Endpoints: handle real-time communication requests like join/leave room, send/receive messages, and check presence (with Redis).
  • FastAPI (HTTP): used for the rest
    • File transmission, chat history retrieval, chat room management, user authentication, etc.
    • Endpoints (HTTP POST): handle setup requests like create/delete room.
• FastAPI WebSocket vs. websockets library (Python)
  • FastAPI WebSocket: for server only (allowing endpoints/routes)
    • Common methods: accept(), send_text(), receive_text(), send_bytes(), receive_bytes(), send_json(), receive_json(), close()
    • Common exceptions: WebSocketDisconnect
  • websockets: for client connection to server
    • Common methods: client.connect(), server.serve(), send(), recv(), close()
    • Common exceptions: ConnectionClosed, InvalidHandshake, InvalidMessage, PayloadTooBig, InvalidState, WebSocketException
• NanoID vs. UUID:
  • Nano is shorter, faster to generate, and is best for client-side generation, web URLs, databases where ID size matters. It is however not a formal standard, and requires careful customization for strong uniqueness.
  • UUID is a formal standard, and is best for any cases where strict standardization is required. It is however longer, and often takes more time to generate.
• Docker:
  • To work with Docker containers, often need to create a Dockerfile to define Docker Image, and create a compose.yaml file to define how to run the Image to a Container.