arturohernandez10

Data processing improvements & problems

Rationale, currently the system once it has all data from the external system, does all updates in a single call. If anything fails, we would execute the entire call again. If another relevant transaction happens in the interim time, we don't have a way to handle it. The large task record created, with spurious properties, makes it difficult to trace and debug.

All reads and updates are done in a single call.
The large transaction code, is partially idempotent, but not fully.
We don't have a way to handle out of order events.
We don't have a way to retry errors.
Traceability is not good.
There is no queue monitoring.

`monitorUsage` Function Documentation

Overview

The monitorUsage function is designed to inspect and report on the memory usage of specific objects in a Node.js application, particularly those marked with a unique identifier. This tool aims to provide developers with an insight into memory allocation and retention tied to marked cache objects, which can be invaluable for diagnosing potential memory leaks or for optimizing memory usage.

Rationale

In large-scale applications, memory optimization is crucial for performance and scalability. However, determining which objects consume significant memory can be challenging. By marking crucial objects (like cache objects) and using tools like node-heapdump-analyzer, specific insights into these objects' memory footprint can be gained. This function serves as a utility to extract and summarize this information.

I am working on an inline formatting tool for the editorjs opensource component. The objective is to mark portions of text as input or output. The challenge in making such a tool is that we need to deal with the nested nature of html. For our objective it is clear that text cannot be input and output nor we can have output inside input.

We already have a base control that works, but it does not handle nesting properly. We need to come up with a process that can be implemented with the surround(range: Range) API call. And we need to describe the different cases that we need to support, considering the user experience. For example, the user right clicks and sees the tool menu with the specific tool highlighted or not. Then the user presses the button. At that point it is up to the code to decide what the behavior's should be.

Taking all of this in mind lets start with the use cases, followed by any useful API that is available and is relevant to out task. Then we can describe logically but as concise as pos

graph TD

%% sinks and sources sUI[[User Input]] sAT[[Activity Tracking]] sPCC[[Prompt Combination and Completion]] sFS[[File System]] sPO[[Prompt Organization]] sUI[[User Interface]] sPS[[Prompt Storage]]

In tne command prompt type

npm init
npm i -D typescript
npx tsc --init
npm i -D jest ts-jest @types/jest
npx ts-jest config:init

Setup of local enviroment authentication

Config of oAuth workflow

In OAuth authentication an application redirects the user to an external site, which offers the user a login screen. When it's done it will redirect back to a specific callback url indicated in the original redirect to google in this case. The application then makes a direct http call to google to exchange the information passed in the callback for an access_token. The access token is used to verify every API request to the backend API. The current implementation is a hybrid flow which means both the client app and the api are part of the authentication flow.

For configuration the application requires:

Setting	Description

SQL Server temporal table alternative with only true changes

Background

SQL Server temporal tables provides a simple way to store historical data. Without a lot of extra effort. The one downside is that it won't filter out multiple updates with the same values. This code will. And it includes instructions on how to produce multiple of these scripts with little effort.

Run dictionary query in SQL Server and copy the JSON output.

How to create and distribute a new Javascript library

In this guide we use a really good typescript library starter package, with the nondescript name of typescript-starter. Since typescript is a superset of javascript with better IDE support. It includes CI/CD configuration, publishing, documentation and release notes documentation. So it is a great start.

Project and repository setup

Run the following command line and follow the prompts. Make sure to name the project with this convention @{{OWNER}}/library-name

npx typescript-starter

Installing and setting up papercut via command line

You need to have choco installed if you don't, run the following in a powershell with admin privileges.

Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))
choco install papercut
echo 127.0.0.1  smtp.sendgrid.net >> %WINDIR%\System32\Drivers\Etc\Hosts

If you already have choco just run the following commands in any console, with admin rights

	@echo off
	setlocal enabledelayedexpansion
	set /p projectName=Project name:
	mkdir %projectName%
	cd %projectName%
	cmd /c npm init -y
	cmd /c npm install --save-dev typescript jest @types/jest @babel/preset-typescript
	cmd /c jest --init
	echo { > tsconfig.json
	echo " \"compilerOptions\": {" >> tsconfig.json