🪢 Added resolvers for LoCG

This commit is contained in:
2026-03-04 23:36:10 -05:00
parent cad3326417
commit b753481754
10 changed files with 3906 additions and 253 deletions

337
README_SCHEMA_STITCHING.md Normal file
View File

@@ -0,0 +1,337 @@
# GraphQL Schema Stitching - Unified Gateway
This service now implements **GraphQL Schema Stitching** to combine multiple GraphQL schemas into a single unified endpoint.
## Architecture
```
┌─────────────────────────────────────────────────────────────┐
│ Client Application │
└────────────────────────┬────────────────────────────────────┘
│ Single GraphQL Endpoint
┌─────────────────────────────────────────────────────────────┐
│ API Gateway (port 3080) │
│ /graphql endpoint │
└────────────────────────┬────────────────────────────────────┘
│ Moleculer RPC
┌─────────────────────────────────────────────────────────────┐
│ Gateway Service (Schema Stitching) │
│ - Combines local + remote schemas │
│ - Routes queries to appropriate service │
└────────┬────────────────────────────────────┬───────────────┘
│ │
│ Local Schema │ Remote Schema
▼ ▼
┌────────────────────┐ ┌────────────────────────┐
│ Local Services │ │ Remote GraphQL Server │
│ - ComicVine │ │ (port 3000) │
│ - Metron │ │ - Core Service │
│ - Metadata │ │ - Other queries │
└────────────────────┘ └────────────────────────┘
```
## What is Schema Stitching?
Schema stitching combines multiple GraphQL schemas into a single unified schema. This allows you to:
1. **Query multiple services** through a single endpoint
2. **Combine data** from different sources in one request
3. **Maintain service independence** while providing a unified API
4. **Gradually migrate** services without breaking existing clients
## Configuration
### Environment Variables
Set the remote GraphQL server URL:
```bash
export REMOTE_GRAPHQL_URL="http://localhost:3000/graphql"
```
If not set, it defaults to `http://localhost:3000/graphql`.
### Service Files
- **[`services/gateway.service.ts`](services/gateway.service.ts)** - Gateway service with schema stitching logic
- **[`services/api.service.ts`](services/api.service.ts)** - API gateway routing to the gateway service
- **[`services/graphql.service.ts`](services/graphql.service.ts)** - Original local GraphQL service (still available)
## How It Works
### 1. Schema Introspection
On startup, the gateway service:
1. Introspects the remote GraphQL server at port 3000
2. Builds a client schema from the introspection result
3. Creates a local executable schema from the metadata service
4. Stitches both schemas together
### 2. Query Routing
When a query is received:
1. The gateway analyzes the query
2. Routes local queries (ComicVine, Metron) to local resolvers
3. Routes remote queries to the remote GraphQL server
4. Combines results if the query spans both schemas
### 3. Fallback Behavior
If the remote server is unavailable:
- The gateway starts with **local schema only**
- Logs a warning about remote unavailability
- Continues to serve local queries normally
- Remote queries will fail gracefully
## Usage
### Starting the Service
```bash
npm run dev
```
The unified GraphQL endpoint will be available at: `http://localhost:3080/graphql`
### Example: Local Query (Metadata Service)
```graphql
query SearchComicVine {
searchComicVine(input: {
query: "Batman",
resources: "volume",
limit: 5
}) {
number_of_total_results
results {
id
name
start_year
}
}
}
```
### Example: Remote Query (Core Service on port 3000)
Assuming your remote server has queries like `getUser`, `getComics`, etc.:
```graphql
query GetUser {
getUser(id: "123") {
id
username
email
}
}
```
### Example: Combined Query (Both Services)
```graphql
query CombinedQuery {
# Local metadata service
searchComicVine(input: {
query: "Batman",
resources: "volume",
limit: 3
}) {
results {
id
name
}
}
# Remote core service
getUser(id: "123") {
id
username
}
}
```
## Benefits
### 1. Single Endpoint
- Clients only need to know about one GraphQL endpoint
- Simplifies frontend configuration
- Easier to manage authentication/authorization
### 2. Flexible Queries
- Query data from multiple services in one request
- Reduce network round trips
- Better performance for complex data requirements
### 3. Service Independence
- Each service maintains its own schema
- Services can be developed and deployed independently
- No tight coupling between services
### 4. Gradual Migration
- Add new services without breaking existing clients
- Migrate queries between services transparently
- Maintain backward compatibility
## Monitoring & Debugging
### Logs
The gateway service logs important events:
```
[GATEWAY] Initializing Apollo Gateway with Schema Stitching...
[GATEWAY] Attempting to introspect remote schema at http://localhost:3000/graphql
[GATEWAY] Successfully introspected remote schema
[GATEWAY] Stitching local and remote schemas together...
[GATEWAY] Schema stitching completed successfully
[GATEWAY] Apollo Gateway Server started successfully
```
### Introspection
Query the stitched schema:
```graphql
query IntrospectionQuery {
__schema {
queryType {
name
fields {
name
description
}
}
}
}
```
### Health Check
Check gateway status:
```graphql
query GetGatewayInfo {
__typename
}
```
## Troubleshooting
### Remote Server Unavailable
**Symptom**: Warning logs about remote schema introspection failure
**Solution**:
1. Ensure the remote server is running on port 3000
2. Check the `REMOTE_GRAPHQL_URL` environment variable
3. Verify network connectivity
4. The gateway will continue with local schema only
### Query Routing Issues
**Symptom**: Queries to remote service fail or return null
**Solution**:
1. Check that the remote server is responding
2. Verify the query syntax matches the remote schema
3. Use introspection to see available fields
4. Check gateway logs for routing errors
### Type Conflicts
**Symptom**: Errors about duplicate types or conflicting definitions
**Solution**:
1. Ensure type names are unique across schemas
2. Use schema transformation if needed
3. Consider renaming conflicting types in one schema
4. Check the `mergeTypes` configuration in [`gateway.service.ts`](services/gateway.service.ts)
## Advanced Configuration
### Custom Executors
Modify the executor in [`gateway.service.ts`](services/gateway.service.ts:95) to add:
- Authentication headers
- Request logging
- Error handling
- Caching
### Schema Transformations
Use `@graphql-tools/wrap` to transform schemas:
- Rename types
- Filter fields
- Add custom directives
- Modify field arguments
### Performance Optimization
Consider implementing:
- **DataLoader** for batching requests
- **Response caching** at the gateway level
- **Query complexity analysis** to prevent expensive queries
- **Rate limiting** per client or query type
## Migration from Separate Endpoints
### Before (Separate Endpoints)
```typescript
// Frontend code
const metadataClient = new ApolloClient({
uri: 'http://localhost:3080/graphql'
});
const coreClient = new ApolloClient({
uri: 'http://localhost:3000/graphql'
});
```
### After (Unified Gateway)
```typescript
// Frontend code
const client = new ApolloClient({
uri: 'http://localhost:3080/graphql' // Single endpoint!
});
```
## Comparison with Apollo Federation
| Feature | Schema Stitching | Apollo Federation |
|---------|------------------|-------------------|
| Setup Complexity | Moderate | Higher |
| Service Independence | Good | Excellent |
| Type Merging | Manual | Automatic |
| Best For | Existing services | New microservices |
| Learning Curve | Lower | Higher |
## Related Documentation
- [GraphQL API Documentation](README_GRAPHQL.md)
- [Architecture Overview](ARCHITECTURE.md)
- [Main README](README.md)
## Support
For issues or questions:
1. Check the gateway service logs
2. Verify both servers are running
3. Test each service independently
4. Review the schema stitching configuration
## Future Enhancements
- [ ] Add authentication/authorization at gateway level
- [ ] Implement DataLoader for batching
- [ ] Add response caching
- [ ] Implement query complexity analysis
- [ ] Add rate limiting
- [ ] Support for GraphQL subscriptions
- [ ] Schema transformation utilities
- [ ] Automated schema versioning

196
models/graphql/resolvers.ts Normal file
View File

@@ -0,0 +1,196 @@
/**
* GraphQL Resolvers for ThreeTwo Metadata Service
* Maps GraphQL queries to Moleculer service actions
*/
export const resolvers = {
Query: {
/**
* Search ComicVine for volumes, issues, characters, etc.
*/
searchComicVine: async (_: any, { input }: any, context: any) => {
const { broker } = context;
if (!broker) {
throw new Error("Broker not available in context");
}
return broker.call("comicvine.search", {
query: input.query,
resources: input.resources,
format: input.format || "json",
sort: input.sort,
field_list: input.field_list,
limit: input.limit?.toString(),
offset: input.offset?.toString(),
});
},
/**
* Advanced volume-based search with scoring and filtering
*/
volumeBasedSearch: async (_: any, { input }: any, context: any) => {
const { broker } = context;
if (!broker) {
throw new Error("Broker not available in context");
}
const result = await broker.call("comicvine.volumeBasedSearch", {
query: input.query,
resources: input.resources,
format: input.format || "json",
limit: input.limit,
offset: input.offset,
fieldList: input.fieldList,
scorerConfiguration: input.scorerConfiguration,
rawFileDetails: input.rawFileDetails,
});
// Transform the result to match GraphQL schema
return {
results: result.results || result,
totalResults: result.totalResults || result.length || 0,
};
},
/**
* Get volume details by URI
*/
getVolume: async (_: any, { input }: any, context: any) => {
const { broker } = context;
if (!broker) {
throw new Error("Broker not available in context");
}
return broker.call("comicvine.getVolumes", {
volumeURI: input.volumeURI,
fieldList: input.fieldList,
});
},
/**
* Get all issues for a series by comic object ID
*/
getIssuesForSeries: async (_: any, { comicObjectId }: any, context: any) => {
const { broker } = context;
if (!broker) {
throw new Error("Broker not available in context");
}
return broker.call("comicvine.getIssuesForSeries", {
comicObjectId,
});
},
/**
* Get generic ComicVine resource (issues, volumes, etc.)
*/
getComicVineResource: async (_: any, { input }: any, context: any) => {
const { broker } = context;
if (!broker) {
throw new Error("Broker not available in context");
}
return broker.call("comicvine.getResource", {
resources: input.resources,
filter: input.filter,
fieldList: input.fieldList,
});
},
/**
* Get story arcs for a volume
*/
getStoryArcs: async (_: any, { volumeId }: any, context: any) => {
const { broker } = context;
if (!broker) {
throw new Error("Broker not available in context");
}
return broker.call("comicvine.getStoryArcs", {
volumeId,
});
},
/**
* Get weekly pull list from League of Comic Geeks
*/
getWeeklyPullList: async (_: any, { input }: any, context: any) => {
const { broker } = context;
if (!broker) {
throw new Error("Broker not available in context");
}
const locgResponse = await broker.call("comicvine.getWeeklyPullList", {
startDate: input.startDate,
currentPage: input.currentPage.toString(),
pageSize: input.pageSize.toString(),
});
// Transform LOCG response to match GraphQL schema
return {
result: locgResponse.result.map((item: any) => ({
name: item.issueName,
publisher: item.publisher,
url: item.issueUrl,
cover: item.coverImageUrl,
description: item.description || null,
price: item.price || null,
rating: item.rating || null,
pulls: item.pulls || null,
potw: item.potw || null,
publicationDate: item.publicationDate || null,
})),
meta: locgResponse.meta,
};
},
/**
* Fetch resource from Metron API
*/
fetchMetronResource: async (_: any, { input }: any, context: any) => {
const { broker } = context;
if (!broker) {
throw new Error("Broker not available in context");
}
const result = await broker.call("metron.fetchResource", {
resource: input.resource,
method: input.method,
query: input.query,
});
return {
data: result,
status: 200,
};
},
},
Mutation: {
/**
* Placeholder for future mutations
*/
_empty: (): null => null,
},
// Custom scalar resolver for JSON
JSON: {
__parseValue(value: any): any {
return value;
},
__serialize(value: any): any {
return value;
},
__parseLiteral(ast: any): any {
return ast.value;
},
},
};

357
models/graphql/typedef.ts Normal file
View File

@@ -0,0 +1,357 @@
import { gql } from "graphql-tag";
/**
* GraphQL Type Definitions for ThreeTwo Metadata Service
* Covers ComicVine and Metron API endpoints
*/
export const typeDefs = gql`
# ============================================
# ComicVine Types
# ============================================
# Image URLs for various sizes
type ImageUrls {
icon_url: String
medium_url: String
screen_url: String
screen_large_url: String
small_url: String
super_url: String
thumb_url: String
tiny_url: String
original_url: String
image_tags: String
}
# Publisher information
type Publisher {
id: Int
name: String
api_detail_url: String
}
# Volume information
type Volume {
id: Int!
name: String!
api_detail_url: String
site_detail_url: String
start_year: String
publisher: Publisher
count_of_issues: Int
image: ImageUrls
description: String
deck: String
}
# Issue information
type Issue {
id: Int!
name: String
issue_number: String
api_detail_url: String
site_detail_url: String
cover_date: String
store_date: String
volume: Volume
image: ImageUrls
description: String
person_credits: [PersonCredit!]
character_credits: [CharacterCredit!]
team_credits: [TeamCredit!]
location_credits: [LocationCredit!]
story_arc_credits: [StoryArcCredit!]
}
# Person credit (writer, artist, etc.)
type PersonCredit {
id: Int
name: String
api_detail_url: String
site_detail_url: String
role: String
}
# Character credit
type CharacterCredit {
id: Int
name: String
api_detail_url: String
site_detail_url: String
}
# Team credit
type TeamCredit {
id: Int
name: String
api_detail_url: String
site_detail_url: String
}
# Location credit
type LocationCredit {
id: Int
name: String
api_detail_url: String
site_detail_url: String
}
# Story arc credit
type StoryArcCredit {
id: Int
name: String
api_detail_url: String
site_detail_url: String
deck: String
description: String
image: ImageUrls
}
# ComicVine search result
type ComicVineSearchResult {
error: String!
limit: Int!
offset: Int!
number_of_page_results: Int!
number_of_total_results: Int!
status_code: Int!
results: [SearchResultItem!]!
}
# Generic search result item (can be volume, issue, etc.)
type SearchResultItem {
id: Int
name: String
api_detail_url: String
site_detail_url: String
image: ImageUrls
description: String
deck: String
# Volume-specific fields
start_year: String
publisher: Publisher
count_of_issues: Int
# Issue-specific fields
issue_number: String
volume: Volume
cover_date: String
}
# Volume-based search result with scoring
type VolumeSearchResult {
volume: Volume!
score: Float
matchedIssues: [Issue!]
}
# Volume-based search response
type VolumeBasedSearchResponse {
results: [VolumeSearchResult!]!
totalResults: Int!
}
# Weekly pull list item (from League of Comic Geeks)
type PullListItem {
name: String
publisher: String
url: String
cover: String
description: String
price: String
rating: Float
pulls: Int
potw: Int
publicationDate: String
}
# Paginated pull list response
type PullListResponse {
result: [PullListItem!]!
meta: PaginationMeta!
}
# Pagination metadata
type PaginationMeta {
currentPage: Int!
totalPages: Int!
pageSize: Int!
totalCount: Int!
hasNextPage: Boolean!
hasPreviousPage: Boolean!
}
# Story arc with enriched data
type StoryArc {
id: Int!
name: String!
deck: String
description: String
image: ImageUrls
issues: [Issue!]
}
# Generic ComicVine resource response
type ComicVineResourceResponse {
error: String!
limit: Int!
offset: Int!
number_of_page_results: Int!
number_of_total_results: Int!
status_code: Int!
results: [SearchResultItem!]!
}
# Volume detail response
type VolumeDetailResponse {
error: String!
status_code: Int!
results: Volume!
}
# Issues for series response
type IssuesForSeriesResponse {
error: String!
limit: Int!
offset: Int!
number_of_page_results: Int!
number_of_total_results: Int!
status_code: Int!
results: [Issue!]!
}
# ============================================
# Metron Types
# ============================================
# Generic Metron resource (flexible JSON response)
scalar JSON
type MetronResponse {
data: JSON
status: Int!
}
# ============================================
# Input Types
# ============================================
# Search parameters
input SearchInput {
query: String!
resources: String!
format: String
sort: String
field_list: String
limit: Int
offset: Int
}
# Volume-based search configuration
input VolumeSearchInput {
query: String!
resources: String!
format: String
limit: Int
offset: Int
fieldList: String
scorerConfiguration: ScorerConfigurationInput
rawFileDetails: JSON
}
# Scorer configuration for matching
input ScorerConfigurationInput {
searchParams: SearchParamsInput
}
# Search parameters for scoring
input SearchParamsInput {
name: String
number: String
year: String
volume: String
}
# Get volumes input
input GetVolumesInput {
volumeURI: String!
fieldList: String
}
# Get resource input
input GetResourceInput {
resources: String!
filter: String
fieldList: String
}
# Weekly pull list input
input WeeklyPullListInput {
startDate: String!
currentPage: Int!
pageSize: Int!
}
# Metron fetch resource input
input MetronFetchInput {
resource: String!
method: String!
query: String
}
# ============================================
# Queries
# ============================================
type Query {
"""
Search ComicVine for volumes, issues, characters, etc.
"""
searchComicVine(input: SearchInput!): ComicVineSearchResult!
"""
Advanced volume-based search with scoring and filtering
"""
volumeBasedSearch(input: VolumeSearchInput!): VolumeBasedSearchResponse!
"""
Get volume details by URI
"""
getVolume(input: GetVolumesInput!): VolumeDetailResponse!
"""
Get all issues for a series by comic object ID
"""
getIssuesForSeries(comicObjectId: ID!): IssuesForSeriesResponse!
"""
Get generic ComicVine resource (issues, volumes, etc.)
"""
getComicVineResource(input: GetResourceInput!): ComicVineResourceResponse!
"""
Get story arcs for a volume
"""
getStoryArcs(volumeId: Int!): [StoryArc!]!
"""
Get weekly pull list from League of Comic Geeks
"""
getWeeklyPullList(input: WeeklyPullListInput!): PullListResponse!
"""
Fetch resource from Metron API
"""
fetchMetronResource(input: MetronFetchInput!): MetronResponse!
}
# ============================================
# Mutations
# ============================================
type Mutation {
"""
Placeholder for future mutations
"""
_empty: String
}
`;

2562
package-lock.json generated

File diff suppressed because it is too large Load Diff

View File

@@ -38,9 +38,15 @@
"telnet-client": "^2.2.5",
"threetwo-ui-typings": "^1.0.14",
"ts-jest": "^25.3.0",
"ts-node": "^8.8.1"
"ts-node": "^10.9.2",
"typescript": "^5.9.3"
},
"dependencies": {
"@graphql-tools/delegate": "^12.0.8",
"@graphql-tools/schema": "^10.0.31",
"@graphql-tools/stitch": "^10.1.12",
"@graphql-tools/utils": "^11.0.0",
"@graphql-tools/wrap": "^11.1.8",
"@types/axios": "^0.14.0",
"@types/jest": "^25.1.4",
"@types/mkdirp": "^1.0.0",
@@ -51,18 +57,20 @@
"delay": "^5.0.0",
"dotenv": "^10.0.0",
"got": "^12.0.1",
"graphql": "^16.13.1",
"graphql-tag": "^2.12.6",
"imghash": "^0.0.9",
"ioredis": "^4.28.1",
"jsdom": "^19.0.0",
"leven": "^3.1.0",
"lodash": "^4.17.21",
"moleculer": "^0.14.28",
"moleculer-apollo-server": "^0.4.0",
"moleculer-web": "^0.10.5",
"nats": "^1.3.2",
"paginate-info": "^1.0.4",
"query-string": "^7.0.1",
"string-similarity": "^4.0.4",
"typescript": "^3.8.3"
"string-similarity": "^4.0.4"
},
"engines": {
"node": ">= 10.x.x"

View File

@@ -56,6 +56,163 @@ export default class ApiService extends Service {
// Enable/disable logging
logging: true,
},
// GraphQL Gateway endpoint with schema stitching
{
path: "/graphql",
whitelist: ["gateway.query"],
cors: {
origin: "*",
methods: ["GET", "POST", "OPTIONS"],
allowedHeaders: ["*"],
exposedHeaders: [],
credentials: false,
maxAge: 3600,
},
aliases: {
"POST /": async (req: any, res: any) => {
try {
const { query, variables, operationName } = req.body;
const result = await req.$ctx.broker.call("gateway.query", {
query,
variables,
operationName,
});
res.setHeader("Content-Type", "application/json");
res.end(JSON.stringify(result));
} catch (error: any) {
res.statusCode = 500;
res.setHeader("Content-Type", "application/json");
res.end(JSON.stringify({
errors: [{
message: error.message,
extensions: {
code: error.code || "INTERNAL_SERVER_ERROR",
},
}],
}));
}
},
"GET /": async (req: any, res: any) => {
// Support GraphQL Playground/introspection via GET
const query = req.$params.query;
const variables = req.$params.variables
? JSON.parse(req.$params.variables)
: undefined;
const operationName = req.$params.operationName;
try {
const result = await req.$ctx.broker.call("gateway.query", {
query,
variables,
operationName,
});
res.setHeader("Content-Type", "application/json");
res.end(JSON.stringify(result));
} catch (error: any) {
res.statusCode = 500;
res.setHeader("Content-Type", "application/json");
res.end(JSON.stringify({
errors: [{
message: error.message,
extensions: {
code: error.code || "INTERNAL_SERVER_ERROR",
},
}],
}));
}
},
},
bodyParsers: {
json: {
strict: false,
limit: "1MB",
},
},
mappingPolicy: "restrict",
logging: true,
},
// Standalone metadata GraphQL endpoint (no stitching)
// This endpoint exposes only the local metadata schema for the core-service to stitch
{
path: "/metadata-graphql",
whitelist: ["graphql.query"],
cors: {
origin: "*",
methods: ["GET", "POST", "OPTIONS"],
allowedHeaders: ["*"],
exposedHeaders: [],
credentials: false,
maxAge: 3600,
},
aliases: {
"POST /": async (req: any, res: any) => {
try {
const { query, variables, operationName } = req.body;
const result = await req.$ctx.broker.call("graphql.query", {
query,
variables,
operationName,
});
res.setHeader("Content-Type", "application/json");
res.end(JSON.stringify(result));
} catch (error: any) {
res.statusCode = 500;
res.setHeader("Content-Type", "application/json");
res.end(JSON.stringify({
errors: [{
message: error.message,
extensions: {
code: error.code || "INTERNAL_SERVER_ERROR",
},
}],
}));
}
},
"GET /": async (req: any, res: any) => {
// Support GraphQL Playground/introspection via GET
const query = req.$params.query;
const variables = req.$params.variables
? JSON.parse(req.$params.variables)
: undefined;
const operationName = req.$params.operationName;
try {
const result = await req.$ctx.broker.call("graphql.query", {
query,
variables,
operationName,
});
res.setHeader("Content-Type", "application/json");
res.end(JSON.stringify(result));
} catch (error: any) {
res.statusCode = 500;
res.setHeader("Content-Type", "application/json");
res.end(JSON.stringify({
errors: [{
message: error.message,
extensions: {
code: error.code || "INTERNAL_SERVER_ERROR",
},
}],
}));
}
},
},
bodyParsers: {
json: {
strict: false,
limit: "1MB",
},
},
mappingPolicy: "restrict",
logging: true,
},
],
// Do not log client side errors (does not log an error response when the error.code is 400<=X<500)
log4XXResponses: false,

View File

@@ -220,29 +220,70 @@ export default class ComicVineService extends Service {
"passed to fetchVolumesFromCV",
ctx.params
);
// Send initial status to client
await this.broker.call("socket.broadcast", {
namespace: "/",
event: "CV_SCRAPING_STATUS",
args: [
{
message: `Starting volume search for: ${ctx.params.scorerConfiguration.searchParams.name}`,
stage: "fetching_volumes"
},
],
});
const volumes = await this.fetchVolumesFromCV(
ctx.params,
results
);
// Notify client that volume fetching is complete
await this.broker.call("socket.broadcast", {
namespace: "/",
event: "CV_SCRAPING_STATUS",
args: [
{
message: `Fetched ${volumes.length} volumes, now ranking matches...`,
stage: "ranking_volumes"
},
],
});
// 1. Run the current batch of volumes through the matcher
const potentialVolumeMatches = rankVolumes(
volumes,
ctx.params.scorerConfiguration
);
// Sort by totalScore in descending order to prioritize best matches
potentialVolumeMatches.sort((a: any, b: any) => b.totalScore - a.totalScore);
// Notify client about ranked matches
await this.broker.call("socket.broadcast", {
namespace: "/",
event: "CV_SCRAPING_STATUS",
args: [
{
message: `Found ${potentialVolumeMatches.length} potential volume matches, searching for issues...`,
stage: "searching_issues"
},
],
});
// 2. Construct the filter string
// 2a. volume: 1111|2222|3333
let volumeIdString = "volume:";
potentialVolumeMatches.map(
(volumeId: string, idx: number) => {
(volumeMatch: any, idx: number) => {
if (
idx >=
potentialVolumeMatches.length - 1
) {
volumeIdString += `${volumeId}`;
volumeIdString += `${volumeMatch.id}`;
return volumeIdString;
}
volumeIdString += `${volumeId}|`;
volumeIdString += `${volumeMatch.id}|`;
}
);
@@ -286,6 +327,39 @@ export default class ComicVineService extends Service {
console.log(
`Total issues matching the criteria: ${issueMatches.data.results.length}`
);
// Handle case when no issues are found
if (issueMatches.data.results.length === 0) {
await this.broker.call("socket.broadcast", {
namespace: "/",
event: "CV_SCRAPING_STATUS",
args: [
{
message: `No matching issues found. Try adjusting your search criteria.`,
stage: "complete"
},
],
});
return {
finalMatches: [],
rawFileDetails,
scorerConfiguration,
};
}
// Notify client about issue matches found
await this.broker.call("socket.broadcast", {
namespace: "/",
event: "CV_SCRAPING_STATUS",
args: [
{
message: `Found ${issueMatches.data.results.length} issue matches, fetching volume details...`,
stage: "fetching_volume_details"
},
],
});
// 3. get volume information for the issue matches
if (issueMatches.data.results.length === 1) {
const volumeInformation =
@@ -299,9 +373,44 @@ export default class ComicVineService extends Service {
);
issueMatches.data.results[0].volumeInformation =
volumeInformation;
return issueMatches.data;
// Notify scoring for single match
await this.broker.call("socket.broadcast", {
namespace: "/",
event: "CV_SCRAPING_STATUS",
args: [
{
message: `Scoring 1 match...`,
stage: "scoring_matches"
},
],
});
// Score the single match
const scoredMatch = await this.broker.call(
"comicvine.getComicVineMatchScores",
{
finalMatches: issueMatches.data.results,
rawFileDetails,
scorerConfiguration,
}
);
// Notify completion
await this.broker.call("socket.broadcast", {
namespace: "/",
event: "CV_SCRAPING_STATUS",
args: [
{
message: `Search complete! Found 1 match.`,
stage: "complete"
},
],
});
return scoredMatch;
}
const finalMatches = issueMatches.data.results.map(
const finalMatchesPromises = issueMatches.data.results.map(
async (issue: any) => {
const volumeDetails =
await this.broker.call(
@@ -315,9 +424,24 @@ export default class ComicVineService extends Service {
return issue;
}
);
// Wait for all volume details to be fetched
const finalMatches = await Promise.all(finalMatchesPromises);
// Notify client about scoring
await this.broker.call("socket.broadcast", {
namespace: "/",
event: "CV_SCRAPING_STATUS",
args: [
{
message: `Scoring ${finalMatches.length} matches...`,
stage: "scoring_matches"
},
],
});
// Score the final matches
const foo = await this.broker.call(
const scoredMatches = await this.broker.call(
"comicvine.getComicVineMatchScores",
{
finalMatches,
@@ -325,14 +449,49 @@ export default class ComicVineService extends Service {
scorerConfiguration,
}
);
return Promise.all(finalMatches);
// Notify completion
await this.broker.call("socket.broadcast", {
namespace: "/",
event: "CV_SCRAPING_STATUS",
args: [
{
message: `Search complete! Returning scored matches.`,
stage: "complete"
},
],
});
return scoredMatches;
} catch (error) {
console.log(error);
console.error("Error in volumeBasedSearch:", error);
// Surface error to UI
await this.broker.call("socket.broadcast", {
namespace: "/",
event: "CV_SCRAPING_STATUS",
args: [
{
message: `Error during search: ${error.message || 'Unknown error'}`,
stage: "error",
error: {
message: error.message,
code: error.code,
type: error.type,
retryable: error.retryable
}
},
],
});
// Re-throw or return error response
throw error;
}
},
},
getComicVineMatchScores: {
rest: "POST /getComicVineMatchScores",
timeout: 120000, // 2 minutes - allows time for image downloads and hash calculations
handler: async (
ctx: Context<{
finalMatches: any[];

324
services/gateway.service.ts Normal file
View File

@@ -0,0 +1,324 @@
import { Service, ServiceBroker } from "moleculer";
import { ApolloServer } from "@apollo/server";
import { stitchSchemas } from "@graphql-tools/stitch";
import { wrapSchema } from "@graphql-tools/wrap";
import { print, getIntrospectionQuery, buildClientSchema } from "graphql";
import { AsyncExecutor } from "@graphql-tools/utils";
import axios from "axios";
import { typeDefs } from "../models/graphql/typedef";
import { resolvers } from "../models/graphql/resolvers";
/**
* GraphQL Gateway Service with Schema Stitching
* Combines the local metadata schema with the remote GraphQL server on port 3000
*/
export default class GatewayService extends Service {
private apolloServer?: ApolloServer;
private remoteGraphQLUrl = process.env.REMOTE_GRAPHQL_URL || "http://localhost:3000/graphql";
public constructor(broker: ServiceBroker) {
super(broker);
this.parseServiceSchema({
name: "gateway",
settings: {
// Gateway endpoint path
path: "/graphql",
remoteGraphQLUrl: process.env.REMOTE_GRAPHQL_URL || "http://localhost:3000/graphql",
},
actions: {
/**
* Execute a GraphQL query through the stitched schema
*/
query: {
params: {
query: "string",
variables: { type: "object", optional: true },
operationName: { type: "string", optional: true },
},
async handler(ctx: any) {
try {
if (!this.apolloServer) {
throw new Error("Apollo Gateway Server not initialized");
}
const { query, variables, operationName } = ctx.params;
this.logger.debug("Executing GraphQL query through gateway:", {
operationName,
variables,
});
const response = await this.apolloServer.executeOperation(
{
query,
variables,
operationName,
},
{
contextValue: {
broker: this.broker,
ctx,
},
}
);
if (response.body.kind === "single") {
return response.body.singleResult;
}
return response;
} catch (error) {
this.logger.error("GraphQL gateway query error:", error);
throw error;
}
},
},
/**
* Get stitched schema information
*/
getSchema: {
async handler() {
return {
message: "Stitched schema combining local metadata service and remote GraphQL server",
remoteUrl: this.settings.remoteGraphQLUrl,
};
},
},
},
methods: {
/**
* Create an executor for the remote GraphQL server
*/
createRemoteExecutor(): AsyncExecutor {
const remoteUrl = this.settings.remoteGraphQLUrl;
const logger = this.logger;
return async ({ document, variables, context }) => {
const query = print(document);
logger.debug(`Executing remote query to ${remoteUrl}`);
try {
const response = await axios.post(
remoteUrl,
{
query,
variables,
},
{
headers: {
"Content-Type": "application/json",
},
timeout: 30000, // 30 second timeout
}
);
return response.data;
} catch (error: any) {
logger.error("Remote GraphQL execution error:", error.message);
// Return a GraphQL-formatted error
return {
errors: [
{
message: `Failed to execute query on remote server: ${error.message}`,
extensions: {
code: "REMOTE_GRAPHQL_ERROR",
remoteUrl,
},
},
],
};
}
};
},
/**
* Initialize Apollo Server with stitched schema
*/
async initApolloGateway() {
this.logger.info("Initializing Apollo Gateway with Schema Stitching...");
try {
// Create executor for remote schema
const remoteExecutor = this.createRemoteExecutor();
// Try to introspect the remote schema
let remoteSchema;
try {
this.logger.info(`Attempting to introspect remote schema at ${this.remoteGraphQLUrl}`);
// Manually introspect the remote schema
const introspectionQuery = getIntrospectionQuery();
const introspectionResult = await remoteExecutor({
document: { kind: 'Document', definitions: [] } as any,
variables: {},
context: {},
});
// Fetch introspection via direct query
const response = await axios.post(
this.remoteGraphQLUrl,
{ query: introspectionQuery },
{
headers: { "Content-Type": "application/json" },
timeout: 30000,
}
);
if (response.data.errors) {
throw new Error(`Introspection failed: ${JSON.stringify(response.data.errors)}`);
}
remoteSchema = buildClientSchema(response.data.data);
this.logger.info("Successfully introspected remote schema");
} catch (error: any) {
this.logger.warn(
`Could not introspect remote schema at ${this.remoteGraphQLUrl}: ${error.message}`
);
this.logger.warn("Gateway will start with local schema only. Remote schema will be unavailable.");
remoteSchema = null;
}
// Create local executable schema
const { makeExecutableSchema } = await import("@graphql-tools/schema");
const localSchema = makeExecutableSchema({
typeDefs,
resolvers: {
Query: {
...resolvers.Query,
},
Mutation: {
...resolvers.Mutation,
},
JSON: resolvers.JSON,
},
});
// Stitch schemas together
let stitchedSchema;
if (remoteSchema) {
this.logger.info("Stitching local and remote schemas together...");
stitchedSchema = stitchSchemas({
subschemas: [
{
schema: localSchema,
executor: async ({ document, variables, context }) => {
// Execute local queries through Moleculer broker
const query = print(document);
const broker = context?.broker || this.broker;
// Parse the query to determine which resolver to call
// For now, we'll execute through the local resolvers directly
const result = await this.executeLocalQuery(query, variables, context);
return result;
},
},
{
schema: remoteSchema,
executor: remoteExecutor,
},
],
mergeTypes: true, // Merge types with the same name
});
this.logger.info("Schema stitching completed successfully");
} else {
this.logger.info("Using local schema only (remote unavailable)");
stitchedSchema = localSchema;
}
// Create Apollo Server with stitched schema
this.apolloServer = new ApolloServer({
schema: stitchedSchema,
introspection: true,
formatError: (error) => {
this.logger.error("GraphQL Gateway Error:", error);
return {
message: error.message,
locations: error.locations,
path: error.path,
extensions: {
code: error.extensions?.code,
stacktrace:
process.env.NODE_ENV === "development"
? error.extensions?.stacktrace
: undefined,
},
};
},
});
await this.apolloServer.start();
this.logger.info("Apollo Gateway Server started successfully");
} catch (error: any) {
this.logger.error("Failed to initialize Apollo Gateway:", error);
throw error;
}
},
/**
* Execute local queries through Moleculer actions
*/
async executeLocalQuery(query: string, variables: any, context: any) {
// This is a simplified implementation
// In production, you'd want more sophisticated query parsing
const broker = context?.broker || this.broker;
// Determine which action to call based on the query
// This is a basic implementation - you may need to enhance this
if (query.includes("searchComicVine")) {
const result = await broker.call("comicvine.search", variables.input);
return { data: { searchComicVine: result } };
} else if (query.includes("volumeBasedSearch")) {
const result = await broker.call("comicvine.volumeBasedSearch", variables.input);
return { data: { volumeBasedSearch: result } };
} else if (query.includes("getIssuesForSeries")) {
const result = await broker.call("comicvine.getIssuesForSeries", {
comicObjectId: variables.comicObjectId,
});
return { data: { getIssuesForSeries: result } };
} else if (query.includes("getWeeklyPullList")) {
const result = await broker.call("comicvine.getWeeklyPullList", variables.input);
return { data: { getWeeklyPullList: result } };
} else if (query.includes("getVolume")) {
const result = await broker.call("comicvine.getVolume", variables.input);
return { data: { getVolume: result } };
} else if (query.includes("fetchMetronResource")) {
const result = await broker.call("metron.fetchResource", variables.input);
return { data: { fetchMetronResource: result } };
}
return { data: null };
},
/**
* Stop Apollo Gateway Server
*/
async stopApolloGateway() {
if (this.apolloServer) {
this.logger.info("Stopping Apollo Gateway Server...");
await this.apolloServer.stop();
this.apolloServer = undefined;
this.logger.info("Apollo Gateway Server stopped");
}
},
},
/**
* Service lifecycle hooks
*/
started: async function (this: any) {
await this.initApolloGateway();
},
stopped: async function (this: any) {
await this.stopApolloGateway();
},
});
}
}

View File

@@ -27,19 +27,15 @@ export default class MetronService extends Service {
console.log(ctx.params);
const results = await axios({
method: "GET",
url: `https://metron.cloud/api/${ctx.params.resource}`,
url: `https://metron.cloud/api/${ctx.params.resource}/`,
params: {
name: ctx.params.query.name,
page: ctx.params.query.page,
},
headers: {
"Authorization": "Basic ZnJpc2hpOlRpdHVAMTU4OA=="
},
auth: {
"username": "frishi",
"password": "Titu@1588"
username: "frishi",
password: "Titu@1588"
}
});
return results.data;
},

View File

@@ -42,16 +42,15 @@ import { isAfter, isSameYear, parseISO } from "date-fns";
const imghash = require("imghash");
export const matchScorer = async (
searchMatches: Promise<any>[],
searchMatches: any[],
searchQuery: any,
rawFileDetails: any
): Promise<any> => {
const scoredMatches: any = [];
try {
const matches = await Promise.all(searchMatches);
for (const match of matches) {
// searchMatches is already an array of match objects, not promises
for (const match of searchMatches) {
match.score = 0;
// Check for the issue name match
@@ -93,7 +92,7 @@ export const rankVolumes = (volumes: any, scorerConfiguration: any) => {
// 2. If there is a strong string comparison between the volume name and the issue name ??
const issueNumber = parseInt(scorerConfiguration.searchParams.number, 10);
const issueYear = parseISO(scorerConfiguration.searchParams.year);
const foo = volumes.map((volume: any, idx: number) => {
const rankedVolumes = volumes.map((volume: any, idx: number) => {
let volumeMatchScore = 0;
const volumeStartYear = !isNil(volume.start_year)
? parseISO(volume.start_year)
@@ -132,18 +131,24 @@ export const rankVolumes = (volumes: any, scorerConfiguration: any) => {
// 3. If issue number falls in the range of candidate volume's first issue # and last issue #, +3 to volumeMatchScore
if (!isNil(firstIssueNumber) && !isNil(lastIssueNumber)) {
if (
firstIssueNumber <= issueNumber ||
firstIssueNumber <= issueNumber &&
issueNumber <= lastIssueNumber
) {
volumeMatchScore += 3;
}
}
if (issueNameMatchScore > 0.5 && volumeMatchScore > 2) {
console.log(`Found a match for criteria, volume ID: ${volume.id}`);
return volume.id;
console.log(`Found a match for criteria, volume ID: ${volume.id}, score: ${volumeMatchScore}, name match: ${issueNameMatchScore.toFixed(2)}`);
return {
id: volume.id,
volumeMatchScore,
issueNameMatchScore,
totalScore: volumeMatchScore + issueNameMatchScore
};
}
return null;
});
return foo.filter((item: any) => !isNil(item));
return rankedVolumes.filter((item: any) => !isNil(item));
};
const calculateLevenshteinDistance = async (match: any, rawFileDetails: any) =>